Quantcast
Channel: Bas Lijten – Bas Lijten
Viewing all 54 articles
Browse latest View live

How to deploy Sitecore web deploy packages using the VSTS Azure App Service task

0
0

With the introduction of the Sitecore 8.2, Sitecore also introduced some Sitecore web deployment packages (WDP), which are used by the Sitecore-Azure-Quickstart-Templates for the deployment of Sitecore. When using ARM templates to provision the Sitecore Infrastructure and deploy the Sitecore application, this works fabulously. But when there is a requirement to use the VSTS Azure App Service deployment task, these packages can’t be used, due to two reasons. This blogpost explains why this task can’t be used and how to fix it (and explains why I spend a lot of time on writing a custom deployment script)

Using the Azure App Service Deploy task

Using the Azure App Service deploy task to deploy the Sitecore baseline has a few benefits: it is maintained by Microsoft and it has a lot of options to configure the (web) deployment of web deploy packages. It offers functionality to deploy to slots, execute xml transformations, use XML variable substitution (which prevents the need for paramameters.xml) and easily set extra arguments, for example to set the parameter values which might be required by the parameters.xml. Lastly, it offers some functions to set App Settings and Configuration Settings on the App Service. All the magic in this task is converted to one beautiful msdeploy command. In other words: it offers a complete set of functions to configure the app service, which should prevent us from writing custom deployment logic.

The first error: “Error: Source (Manifest) and destination (ContentPath) are not compatible for the given operation”

However, when using the default Sitecore cloud wdp’s (which can be downloaded from dev.sitecore.net), the deployment fails miserably:

“Error: Source (Manifest) and destination (ContentPath) are not compatible for the given operation”

The vsts task creates a msdeploy task and for some reason, this vsts task can’t handle the Sitecore web deploy package. This task creates the following msdeploy command:

“msdeploy -verb:sync -source:package=Sitecore_package.zip -dest:ContentPath=”azure-site” -…

In “normal” situations (custom build web deploy packages), this vsts task does it job. Luckily, all vsts tasks are open sourced on github, which means that we could take a peak at the code to find out what went wrong.

Some code exists which determines whether or not a web deployment package has been used. This is determined by the following function:

 

This code checks whether or not the parameters.xml file is available and (3) if there is a file called systemInfo.xml. A few weeks back, when I wrote a custom deployment script because I couldn’t get this deployment script to work, I completely missed the and operator && (phun intended) I missed the fact that there is a third way of writing “systemInfo.xml”.

Let’s take a quick look at the contents of the Sitecore web deployment package:

Did you see the capital S? That’s the sole reason that the error appears! As this vsts task isn’t in use by the ARM deployment, the check on systemInfo.xml OR systeminfo.xml isn’t executed; that’s why that method works. Is it a bug by Sitecore? Or is it a bug by Microsoft? I don’t know, but I failed it at both companies 😉. (Github issue. The Sitecore issue is filed under “issue 505891”). The fix is quite simple: rename the file to systemInfo.xml inside the zip. At the end of the article I’ll link to a powershell module which handles this manual action (and which will fix the second error as well).

The second error: “Source does not support parameter called IIS Web Application Name”

As a problem never comes alone, a second error showed up: “Source does not support parameter called IIS Web Application Name”. When making use of the VSTS task, this IIS Web Application Name is specified as an input parameter during deployment. This can’t omitted in any way, as it is hardcoded in the VSTS task as well:

The only fix is to add this parameter to the parameters.xml.

The fixes: msdeploy (and Rob Habraken) to the rescue

The fastest and most easy fix is to create a new web deploy package. There is a lot of documentation going on regarding this subject, for example, Rob showed in a previous blog on how to do this and he provided a script to create a new package from an existing package.

All I did was modify this script slightly, by adding a parameter called “IIS Web Application Name”. By specifying the argument “-declareParam:name…” the parameter will be added to the parameters.xml in the newly generated package. A free bonus that comes with this script, is that the SystemInfo.xml will be renamed to systemInfo.xml, which means that there is no manual action required anymore to rename that file.

Conclusion

The default Sitecore web deploy package can’t be used with the default vsts Azure App Service task due to two reasons, but the fix is quite easy. This possibility to deploy Sitecore using the Azure App Service task opens up a lot of new possibilities: I will write about these options in a later blogpost.


How to update the default hashing algorithm for Sitecore 9 to SHA512 using msdeploy

0
0

A few years ago I already blogged on updating the default hashing algorithm that Sitecore used. It’s ancient, unsafe and it’s for a change! Luckily, with the introduction of msdeploy packages and the Sitecore on Azure templates, this becomes easy, without having to (manually) change the password after the hashing upgrade and manually changing the hashAlgorithm in the web.config. This can be done by using msdeploy the tool to install and create msdeploy packages.In my previous blogpost I already showed how to add parameters to the web deploy packages, in this approach I am going to show how contents can be changed using this technique. The benefit of this approach is that it can be used with the out of the box Sitecore packages, and, when used as a source for the initial deployment, the right algorithm will be used deploy time and everything will be automated. This method might look a little bit cumbersome, because a manual change might work, but with tens to hundreds of environments, you just want to have everything automated as much as possible.

Source can be found here:

How does the provisioning to Azure work?

The provisioning to Azure works a bit different as opposed to the on-premise deployment. During the initial provisioning, the administrator password can be set. The Sitecore arm templates forces a certain policy and when it’s has been accepted, it is used as parameter for the initial setup. This parameter is specified in the “parameters.xml” within the web deploy package:

In this case the parameter (1) Sitecore Admin New Password will replace the “PlaceHolderForPassword” in the file “SetSitecoreAdminPassword.sql”

When taking a closer look at this sql file, the PlaceHolderForPassword is being used as password. Due to the parameters.xml, this value will be replaced by the parametervalue that is specified during deploymenttime. Another observation is the default Hashing algorithm: “SHA1” is used. This isn’t strange, as the hashing Algorithm that Sitecore uses for years, is SHA1 as well.

This SHA1 value also appears in the web.config

The required changes

As seen in the paragraph above, two changes are needed:

  • The hashing algorithm in the sql script
  • The hashing algorithm in the web.config

As explained in the previous blogpost, the declareparam parameter can be used, to specify a new variable. Depending on the parameter “type” a match is done based on a regexp or XPatch expression.

Modify the sql script

The first change is quite easy: replace SHA1 by SHA2_512.

This is not the only change that is needed for the SQL change: when looking taking a closer look at the sql script, it turned out that the fieldlength for the hashed password was just 20. While this is long enough for a SHA1 password, this isn’t the case for a SHA512 password. Many thanks to Marshall Sorenson for pointing this out on his blog.

This change can be applied in the same way as changing the hashing algorithm:

Please take not of the default value. When using the default value, this value is applied when no parameter value has been provisioned. This comes in handy, as this new package doesn’t break existing provisioning pipelines, as the parameter becomes optional. In other words: when using in conjunction with the existing arm templates, this package will still work and apply the SHA512 changes.

Modify the web.config

The second change isn’t to hard as well, but requires a little bit of XPath magic. The textfile approach could have been used, but as this web.config is an XML-file, the XPath approach is a little bit more appropriate in this case:

Default values

You will probably have taken notice of the “defaultvalue” in these parameters. I specified them for one important reason: your existing ARM or msdeploy scripts will not break. With the defaultvalue being set, this value will be used when the parameter hasn’t been specified. This is definitely the case with the out of the box Sitecore quickstart ARM templates, which means that a change to your existing deployment scripts won’t be needed as well.

 

One big fat “gotcha”

After creating this package, one thing went wrong miserably. At some point in time, the script places temporary values in the new package:

This isn’t a problem for a package that will be re-deployed (this was the general purpose of the script, creating a package that wouldn’t install the existing databases anymore), but for a package that will be used for the initial provisioning of a website, this is killing. The parameters.xml contains some logic to replace, for example, the “PlaceHolderForPassword” strings in the sqlfiles. When the package gets re-created with the value “tmpvalue”, this logic will replace the string “PlaceHolderForPassword” in all sql-files:

This means, that the parameters.xml in the new package, doesn’t work anymore, as the “match” doesn’t exist anymore:

This could easily be solved by adding the following line in the foreach-loop:

It basically checks if there is an exact match (which happens in the TextFile) and uses the original “match” value as the new input value. This will lead to the result where the “match”-value will be replaced by it’s original “match”-value: nothing will change.

 

Summary

Changing the out of the box hashing algorithm is something you should do immediately, but in conjunction with Sitecore on Azure this isn’t the most trivial thing to do, as the password will be set during the provisioning of the environment. While a manual change to the packages looks as the most easy approach, it isn’t the most structural approach. Using a script to do this (and preferably in a separate build in vsts) will lead to a nice reusable package which can be tailored to your own needs.

Federated Authentication in Sitecore – Error: Unsuccessful login with external provider

0
0

I faced this error quite a few times now and I always forget what the root cause of this error was. To keep me away from debugging and reflecting code again I wrote this blogpost 😉When the claim http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier is not present, Sitecore will throw this exception, although a successful login may happen! This blogpost explains the root cause and how to solve the issue

What happens during a federated login?

When someone wants to login using an external identity provider, that person will be redirected to several different places:

  • Redirect to the identity/externallogin pipe, which will handle the correct external identity provider, which will set the right wtrealm et cetera
  • Redirect to the actual identity provider (in our case it’s a double redirect, but that is totally not relevant for the inner workings, but it explains the two redirects in 8) and 10))
  • The identity provider will redirect you to the url specified in your wreply. In our case, we chose to use _sitecoretrust, as we have several systems running under the same domain, where we wanted to have a Single signon integration. More on that in a later blogpost
  • Using that Callbackpath, the actual claimsIdentity is created and all the claim transformations that are specified in your identity provider configuration are applied. The securitytoken will be validated in this step. If this token is not valid, an error will be thrown, otherwise, the user will be redirected to the next step
  • The last step is to redirect back to the /identity/externallogincallback, which will actually do the latest administration to make sure that Sitecore will work correctly.

When getting the message “Unsuccessful login with external provider” comes from “HandleLoginLink” pipeline and this error is generated when there is something wrong with the external login info. One code snippet that will be executed is to check if the identity exists (which is, as the middleware has verified this in step 4), the next one is to validate if the claim http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifieris present. If this is not the case, the error will be thrown, although the external login has been successful.

How to solve this issue

Sitecore offers the possibility to transform claims using rules. This can be done as a shared transformation or as a specific transformation for the identity provider. Make sure to transform an existing, unique claim into this name claim:

The default transformation has been used. If the source claim does not contain a value, than the transformation will always kick in and create a new claim (as defined in the targets) with that same value. If the source does contain a value, than the rule kicks in when both the name as the value are true.

Note: a better solution is to add the claim to the identity provider, if possible. But when you just want to test things out or don’t have any access to the IdP, this solution is a very feasible solution

Summary

This error leads to a wrong assumption, which might make this error hard to solve. Because of the flexible claim transformation rules in Sitecore, it’s very easy to solve this error.

Private Sitecore nuget feeds using VSTS – why we don’t use Sitecore myget and how we work with package management

0
0

First of all: hands down to Sitecore when they created the nuget feed a while back: it’s really, really convenient to be able to use a nuget feed for all those Sitecore packages, including their dependencies. But we had some issues with the way Sitecore versions it’s packages, the fact that we use multiple versions of Sitecore and the way we wanted to provision our own reusable sitecore-specific nuget packages. Aside from that; our existing nuget-feed was a NAS which had many, many performance issues. In the end we came up with a private nuget feed per Sitecore version which contains all the Sitecore assemblies for that specific version, its dependencies and our own reusable nuget packages for that specific Sitecore version.

Source can be found on github

Sitecore versioning

Sitecore versioning on itself is not too bad. When working with Sitecore 9.0 update 1, you basically work with Sitecore 9.0 – revision 171219 – as it was generated on the 19th of December in 2017. When working with nuget, you would have to know that you have to reference the 9.0.171219 version of the nuget packages and not to upgrade to 9.0.2 as it might break things. We still see developers upgrading to the “latest-greatest” version of a package, which would basically put you in unsupported state. This nuget versioning is in no way related to the assembly versioning of Sitecore.

When downloading the Sitecore zip (or web deployment package), it contains all required assemblies. In this package, the Sitecore.Kernel has assemblyversion 11.1.0.0, while Sitecore.Framework.Conditions has assemblyversion 1.1.0.0. Sitecore.EmailCampaign.Analytics does have assemblyversion 6.0.0 and Sitecore.ContentSearch does have assemblyversion 3.1.0.0. They all have a different versioning, which may be related to the functionality; Sitecore probably uses the SymVer method internally.

My expectations on nuget would be that all these assemblies were available under version “9.0.171219”, but it doesn’t look like this is the case. I made a query against all packages, which are 9179 in total. Sitecore 9.0.1 (version 9.0.171219) does have 452 packages, of which 224 are “NoReference” versions.

So what should I do when I would need the Sitecore.Framework.Conditions assemblies for Sitecore 9.0.1? When taking a look at nuget, there are multiple versions:

There is no way I can correlate this to the Sitecore version 9.0.171219 at all. Should I use 3.0.1 as it is the latest version? Or is that version tied to Sitecore 9.0 update 2? Is it safe to use the latest version? Or would I lose my support when using that version?

Inventarisation of nuget packages and their dependencies

Using a powershell script, I did a query on all sitecore packages. This learned me that those 9179 packages had 106 versions tied to it. Some versions can be correlated to specific Sitecore versions, others can’t.

The next step was to do a query on version 9.0.17129 and it’s dependencies. This leads to the a list which can be found here. It gives an overview for all direct dependencies (3rd party and Sitecore) and it’s nuget version that Sitecore refers to. The script which I used can be found here.

In the end, it turned out that there are 306 Sitecore packages and direct depenendies. This knowledge is used to create our own feed!

The dependencygraph of Sitecore is enormous, that I decided just to investigate the first level of dependencies per component. This leads to an enormous graph is well, but it took an acceptable amount of time.

First, make sure that the sitecore-feed has been registered:

Register-PackageSource -Name "sitecore-myget" -Location "https://sitecore.myget.org/F/sc-packages/api/" -ProviderName "nuget"

After registration, the magic (may) happen. First I wrote a recursive script which would get all sitecore packages and its dependencies, all the way down to the very last dependency, but this took a lot of time. So I decided to replace it with only the first level dependencies, which returns “enough” nuget packages.

First, I retrieve all package metadata from the Sitecore myget feed, using the following command:

$packages = Find-Package -Source "sitecore-myget" -AllVersions
$packages901 = $packages | where {$_.Version -eq "$sc_version" }
$packages901WithReferences = $packages901 | where {$_.Name -notlike "*NoReferences"}
$packages901NoReferences = $packages901 | where {$_.Name -like "*NoReferences"}

The next action is to filter all Sitecore 9.0.x versions, in my case this was 9.0.1 (on line 2). In a foreach loop I iterate through all packages which do have references, to install them and get the correct metadata from them.

Using

$pkg = Get-package -Name $Name -RequiredVersion $Version -ErrorAction SilentlyContinue

it can be validated wether or not the package is already installed. If that’s the case, the $pkg is not null and all dependencies can be checked and installed. This installation is important for later upload to VSTS package management. If installation is not required, that line can be disabled.

All dependencies are stored in the form of “parent version.x” -> “child version.y”, for later analysis.
When the package does not exist, it has to be looked up in an external feed using

Find-Package -Name $Name -RequiredVersion $Version -Source $Source-ErrorAction SilentlyContinue

When the package has been found, it can be installed to the local packagemanagement storage.

VSTS package management to the rescue!

With a list of all referenced Sitecore versions, I decided to create a vsts package management feed. It’s not possible to create a direct upstream to Sitecore.myget.org (yet), but there are other ways to fill this feed (scripts provided here). All of the used Sitecore versions, its dependencies are stored in that feed. VSTS package management contains a nice feature, called “Views”. Its normal use is to create several feeds which say anything about the stability of a package (alpha, beta, pre-release, release). When a package has been added to the pre-release feed, it doesn’t show up in the release view (yet) ; This package has to be promoted to the “release” view (which can be done manually or using code. We used the same technique for Sitecore. The difference here is, is that we use the views to reference the correct set of Sitecore nuget packages and its dependencies, instead of using them for its “stability-level”:

When taking a look at the “default” view, a list of all the latest Sitecore versions is visible:

Please take notice of the “HtmlAgilityPack”, as it doesn’t have a view specified, while the “Sitecore.Abstractions.NoReferences” package has version “9.0.180604” and the view “@Release-9.0-update-2” specified. I also uploaded the “9.0.191217” version, but as it has a lower version in this “Local” feed, it’s not directly visible.

When taking a look at the the versioninfo,  both versions can be seen:

When taking a look at the View “Release-9.0-update-1”, the 9.0.180604 version is missing; only the 9.0.171219 version is available in here. Please take note of the HtmlAgilityPack; it’s not available at all in this feed:

Using the User interface, packages can be promoted to different views:

In this “Promote this package” window, the correct view can be selected:

After selecting the view, the package has been made available to that specific view, which is available as a regular nuget feed.

This feed contains all the relevant packages for the Sitecore version and its dependencies. We decided to provision our generic components to this feed as well, so every building block for a certain version, is available from that feed. Our developers will not be able to (accidentally) upgrade to a version that is not supported.

From a Visual Studio client, the nuget package management displays the following package versions for the Sitecore.Abstractions.NoReferences package. At the left, the “general” feed is shown, at the right, the two specific views for Sitecore 9.0 update-1 and Sitecore 9.0 update-2:

Upgrading your code to a new version

This approach makes upgrading to “a” version a blast. By changing your Sitecore feed to the “new” Sitecore version feed (from 9.0.1 tot 9.0.2), all you have to do is upgrading all packages at once and you are (or should) be ready to go:

Summary

Sitecore versions may lead to unwanted actions by developers (for example, upgrading to the latest sitecore version where you are still running 8.2) and in a lot of cases it’s just unclear what version of a package to use. After analyzing the Sitecore feed, its versions and depedencies, we came up a list of all packages that are “tied” to a certain sitecore version. To make it more convenient for our developers, we decided to create an own sitecore feed, with all Sitecore nuget packages and their dependencies, and made just the small subset of packages which are “tied together” available through different views on this nuget feed.

My Sitecore Symposium session – World’s fasted delivery pipeline for Sitecore on Azure

0
0

On October 9th I presented for the 3rd time at the Sitecore Symposium. After a security session, our session with Robbie the Robot, I chose to give some insights into our delivery pipeline for Sitecore on Azure. In this session, I will share my road to this presentation.

This time, I had to hold my presentation on the very first day of the Symposium, which meant that afterwards, I could enjoy all the other presentations. Although I speak quite often and I am pretty confident that I know what I am talking about, I have to admit that I still get a little bit nervous before I have to go on stage and I continue to work on my presentation until the last hour before that presentation, just to be sure that it is perfect.

Getting ready for the presentation

When working on the presentation, I always start with a goal: what do I want to share with the audience? What should the audience know, when I finished my presentation? Based on that goal, I try to setup a story around that goal.

This time, it was very, very hard. Although I was thinking that I was having a hard time with the security and Robbie session, it turned out that this was really the hardest presentation to create, ever. There was so much content that I wanted (and had to) share upon reaching my goal, that I really didn’t know anymore what was relevant and what not. And the hardest part: The structure, the order of the subjects. And it had to fit in 45 minutes.

When working on a presentation, the content that you are going to create is always way too much. The trick in here is to reduce it to a few important parts and try to explain those steps as easy as possible.

I am also a great fan of giving a live demo, but for this presentation, I decided to only use slides with screenshots. Why?

  • Live demo’s take up a lof of time – screenshots are much quicker
  • My demo was very dependent on internet and Azure DevOps (which was down the day before)
  • Screenshots are a great reference when handing out slides
  • Presenter nodes could be added to the screenshots – they help with the story during the presentation

And that that is exactly the part where I get a bit uncertain about the presentation. Altough the content is great, the goal is great and the outcome is great, there always is a feeling of discomfort, because of the reduced steps and complexity and the lack of a live demo, it feels like a mediocre story, where people don’t learn anything new, as it sounds that simple. This is due to all the time spend on this presentation; Every time you spend time on a subject, it gets a little bit more “normal”, until you reach the point where you think that the subject is so basic which let’s you think that everyone is already doing that.

Going on stage

The moment is there; there is a little bit of tension and I head to the room. In this room, there are already 10 people waiting for my presentation. The AV-guys are still away, because they had a break, so I decided to plugin my laptop myself. I am lucky: no hassle with the video, as the large screens and the confidence monitors are working immediately!

Those confidence monitors are GREAT! As I have a lot of slides, without a live demo, I don’t feel comfortable to just stand behind a small desk. This sounds strange, as a lot of people feel safer behind a desk, but I always like to use the complete stage; it allows me to walk the tension out of my body. As the podium was large and there were confidence monitors at both sides of the stage, I was able to freely walk around. A big plus!

People walking in

More and more people are entering the room. As the room is quite far from the partner pavilion, it takes some time before everyone entered the room. I decided to wait a few minutes before I really start. The room is packed, people had to stand and afterwards I heard that some people were not allowed to enter the room anymore, as it was too full.

Ready for Action!

it’s 2:47pm and I decided it’s time to start. Al the tension and nerves that were there, were suddenly gone! A lot of known faces amongst the audience, and just a few people playing around with their phone. On every new slide, people take pictures. This is a good thing; they are interested, listening, which gives me even more energy! At 3:30pm exactly I finished my presentation. Just one person who left the room (and I really don’t care; people should be able to leave a talk when it is not in line with their interest), which means that I probably did a good job!

Summary

During preparations and just before the presentation, I always (and I think a lot of other speakers as well) get nervous, uncertain, but in the end, it’s always fun to talk about the things that you did, discovered and want to share with the community. Remember that you know what you are talking about, that there are always people who know things better, but the majority is there to learn something from you.

The presentation can be downloaded here

Automate your pipeline part 1: World’s fastest delivery pipeline for Sitecore on Azure

0
0

On October 9th I presented for the 3rd time at the Sitecore Symposium. In my previous blogpost I described shared how I felt during the creation of this presentation and on the day itself. In this series of blogposts I’ll describe every subject I discussed during my presentation, which will, in the end, enable you to setup your own fully automatic deployment pipeline using standard Microsoft technologie such as msbuild, msdeploy, nuget and Azure DevOps. This blogpost is just a container for all the upcoming blogposts (which is subject to change). When you are missing a subject, feel free to get in touch with me.

This blogpost is part 1 of the series “Automate your pipeline”. The blogposts may or may not be released in the order as posted in the list below,

  • part 1: Automate your pipeline part 1: World’s fastest delivery pipeline for Sitecore on Azure (this blogpost)
  • part 2: Provisioning vs deployment
  • part 3: different types of modules
  • part 4: automate msdeploy all the things!
  • part 5: create your own packages using msdeploy – the Sitecore baseline
  • part 6: Patch the unpatchable – how to handle changes to the web.config
  • part 7: how to build your business application
  • part 8: how to deploy your business application
  • part 9: speed up your deployments – parallel deployments
  • part 10: speed up your deployments – unicorn
  • part 11: speed up your deployments – does size matter?
  • part 12: deploy with the speed of light
  • part 13: release toggles (release toggles)

Purpose of this series

The main purpose of this blogpost is to share all knowledge that we have build up during our journey to a fully, fast, automated CI/CD pipeline. We prefer just to make use of default Microsoft technology, which means we try use msbuild, msdeploy, Azure DevOps and powershell wherever possible and applicable, without making it overly complex.

The original presentation can be found here

Speed up your deployments – parallel app service deployments in Azure DevOps

0
0

Note: Although this blogpost series is focused towards deploying Sitecore with the speed of light, all the information in this blogpost regular web applications and app service deployments

Deploying multiple web deployment packages to multiple app services may take some time. Where parallel jobs in the Build are possible, this is not possbile (yet) in Azure. ARM templates could be used (but I am not 100% sure), but we chose to use app service deployments, as it gives us much more flexibility.

This blogpost is part 9 of the series “Automate your pipeline”. The blogposts may or may not be released in the order as posted in the list below,

  • part 1: Automate your pipeline part 1: World’s fastest delivery pipeline for Sitecore on Azure
  • part 2: Provisioning vs deployment
  • part 3: different types of modules
  • part 4: automate msdeploy all the things!
  • part 5: create your own packages using msdeploy – the Sitecore baseline
  • part 6: Patch the unpatchable – how to handle changes to the web.config
  • part 7: how to build your business application
  • part 8: how to deploy your business application
  • part 9: speed up your deployments – parallel app service deployments in Azure DevOps using the app Service deploy task(this blogpost)
  • part 10: speed up your deployments – unicorn
  • part 11: speed up your deployments – does size matter?
  • part 12: deploy with the speed of light
  • part 13: release toggles (release toggles)

Deployments to the Azure App Service using msdeploy

MSDeploy is used to deploy web deployment packages to azure app services. ARM uses this tool under the hood to deploy the defined web deploy packages, the Azure App Service deploy task can use msdeploy and, of course, PowerShell could be used. We chose to use the Azure App Service task, as the ARM templates don’t give too much flexibility from a msdeploy perspective; several actions that can be configured using the msdeploy command or Azure App Service task are not implemented in ARM, for example the -skip: options. When your package(s) require new parameters, the ARM templates have to be changed as well. We try to keep our ARM templates generic; this way, the can be reused amongst different projects, that’s why we don’t prefer to change those ARM templates into application specific templates. As the Azure App Service deploy task does a lot of the heavy lifting, such as setting up the connection, getting credentials, we prefer to use the App Service task instead of creating an own powershell task and maintain that one.

Release jobs

Within a release pipeline different jobs can be defined. With a job a set of tasks are configured, which are run sequentially. In our situation, we have to deploy multiple app services to multiple regions at once. All these app services are part of a CMS platform called “Sitecore” and they all fullfill different roles; the “CD” role serves the role of “Content Delivery”  to the visitor, the “CM” role serves the role where content can be managed.

Parallel deployments using agent jobs are not possible in the release pipeline

At the moment of writing, 18/10/2018, parallel jobs are not yet possible in the Azure DevOps deployment pipeline. With other words, it’s not possible to define a job which deploys an app service in region “Northern Europe”, define another job which deploys to an app service in region “Western Europe” and to run these jobs simultaneously. However, this is possible in the Azure DevOps build pipeline: By selecting the correct dependencies, Azure DevOps decides which jobs can run in parallel and which not, but, this option is not yet available within the release pipelines:

How to configure parallel deployments of Azure App Service

Although parallel jobs are not possible, it is possible to run tasks in parallel, but it requires a bit of configuration, which will be explained in this section.

In the image below a simplified Sitecore setup is displayed, where all services need to be depoyed at once. (For zero downtime we are using staging slots, often referred to as “blue-green deployments”)

simplified sitecore diagram

simplified sitecore diagram

The first step is to define each task in the pipeline. Take note of the naming convention: We chose to name what role will be deployed to what specific region. This will help identifying what’s happening in a later stage.

Each task has a unique operation to:

  • deploy CD role to the West Europe region
  • deploy CM role to the West Europe region
  • deploy CD role to the north Europe region
  • deploy CD role to the North Europe region

The second step, is to define the unique actions. In my case, I have the apps with role “CM, CD” and regions “West, North”, which could be configuration parameters. these tasks could (and should) be run in parallel.

The trick lies within configuring “Multi-configuration” with the “Execution plan” section and setting the parameters as multipliers. This can be configured on the job itself.

Execution plan in Azure DevOps

Execution plan in Azure DevOps

To get this multiplier to work, two variables, called “role” and “region” should be added. Please mention that by adding regions or roles, extra combinations of these configurations will be added (this is the case in the production setup):

Variables in Azure DevOps release pipeline

Variables in Azure DevOps release pipeline

When this releases is queued, every task would be executed, for every configuration, which means that every app service would have been deployed 4 times and sequentually, which means, things will go wrong and will take longer. T

The trick resides into configuring a “custom condtion”. This is available within the Control options section. change the “Run this task” option from “Only when all previous tasks have succeeded” to “Custom Conditions”. An option appears to specify a state when this task should be run. “Only when configuration parameters equal to the specified role and region”. In every other situation, the task will be skipped.

the custom condition that should be set is the following, where the ‘west’ and ‘CD’ value change for every task. The complete syntax can be found here.

and(eq(variables['region'],'west'), eq(variables['role'],'CD'))

Custom Condtion in Azure DevOps

Save the release and queue it. When running this release, Azure DevOps will add a few extra tabs in which every configuration is displayed. Within this tab, every task is shown. While each configuration is running in parallel, the tasks that should not run within that job are skipped as well:

Azure DevOps parallel tasks

Azure DevOps parallel tasks

 

Summary

Although the convenient way of running jobs in parallel is not yet availalbe for the release pipelines in Azure DevOps, it is possible to get this to work. In case of the sitecore platform, where sometimes more than 18 apps have to be deployed in parallel, this can save a lot of time and will dramatically increase the speed of your acceptance and production deployments

 

Warmup your application on Azure App service when scaling up and swapping slots using “Application Initialization”

0
0

A common problem on Azure web apps when scaling up or swapping slots is “stuttering”. At the moment an instance is added to the pool (scale out) or your swap is swapped (reload the app on the slot), your application is “cold , which means that your application on that instance needs to be reloaded. In the case of Sitecore (or other large applications), this may take a while. In this period, visitors may face a long loading time, which may take up to a few minutes.

This stuttering can easily be resolved for Azure App Services, making use of default mechanisms which isn’t widely used or known. This mechanism is called “Application Initialization“, which was introduced in IIS version 8:

“Application Initialization can start the initialization process automatically whenever an application is started.”

This process doesn’t make the startup process faster, but starts the process sooner.  The fun part is: when making use of scaling within Azure Apps, this mechanism can be used to warmup the specific new azure app service instances.

Within the web.WebServer section of the web.config, the Applicationinitialization node may be added. Within this node, the pages that need to be warmed can be specified over here:

<web.webServer>
   <applicationInitialization>
      <add initializationPage="/" />
      <add initializationPage="/page-2" /> 
   </applicationInitialization>
</web.webServer>

As stated previously, this action starts the process sooner, but doesn’t make it faster. This applies to normal IIS as well as Azure App Services. So after a restart, and you’d immediately try to visit that website, you would experience a long loading time. But the Azure App service has a cool feature. When scaling up (manually or by using autoscale) or swapping slots, a new instance is created and started. The initializationPages are being touched internally and Azure App Services waits with the actual swap or addition to the Azure App Service pool until all the pages which have been defined in that section, have been loaded. This will prevent that “cold” applications will not be released, which means that there will be no “stuttering” of the application.

 


Increase your (Sitecore) performance by enabling the local cache on Azure App Services

0
0

Although this blog focuses primarily on Sitecore, this blogpost is applicable for any application which is hosted on an Windows Azure App Service.

The file system,which is used by Azure App Services, always points to D:\home, but it is in fact a link to a (slow) performing share. Sitecore can greatly benefit from enabling a local cache, this blogpost describes how to enable this cache.

Important to know, is that the app service can write to it’s cached folder, but after an application restart, or after a new deployment, all these changes are discarded.

The first step is to set the following application setting:

WEBSITE_LOCAL_CACHE_OPTION = "Always"

this will enable the local cache. It copies the shared wwwroot to the D:\home\wwwroot folder, but it has a limitation: by default, only 300MB will be transferred locally. As Sitecore surpasses this amount easily (a fresh Sitecore 9.1 installation takes over 1Gb of storage), this amount has to be increased. This amount can be altered by adding the following setting:

WEBSITE_LOCAL_CACHE_SIZEINMB = "2000"

The amount of 2000 is the current maximum, but is enough for a Sitecore installation.

When working with a local cache, problems may arise with local stored media items: They are not shared anymore, but there is an excellent solution by Per Osbeck, which can be found over here.

Don’t configure this on your staging slot

A second important note is not to configure this on your staging slot: as you want to test clean deployments, this local cache shouldn’t be enabled on the staging slots, but only on your production slot; this means that you need to create a slot specific setting, which ties to your production environment. The problem in here is that slot specific settings cause an application recycle, but there is a solution for that in my next blogpost – How to warmup your application before swapping slots in blue-green scenarios.

More information regarding this subject can be found on this MSDN page

Warmup your Sitecore on Azure App Service applications when using Slot settings

0
0

Using deployment slots in Azure App Services is a best practice to deploy Sitecore your application with zero downtime. However, there are some drawbacks, for example when slot specific settings are being used. This blogpost describes how to come around these issues.

When using a blue-green deployment zero downtime deployments can be realized for Sitecore. However, when making use of slot specific settings, this may lead to at least stuttering of your application(s). At the moment that a slot is swapped, the slot specific application settings are being transferred to the application, which causes the application to recycle.

In my previous blogpost I wrote about slot specific settings, which enables a fast local cache of the filesystem. As this increased speed is a very nice improvement, the application recycle has to be prevented or it has to be solved in another way.

Default Azure mechanics – swap with preview

A mechanism that should be used, is to use the “Swap with preview” option. This option recycles your application and loads the slot specific settings of your target slot into your app domain, which means; it’s ready to use on production. However, in this swap with preview option, you don’t have a guaranteed warmed up application: swapping to fast means that your application might stutter when it hasn’t been fully loaded yet.

Use application Initialization for a guaranteed warmup

As described in a previous blogpost, the application initialization configuration can be used to define pages that need to be initialized when recycling an application. Azure does have mechanism when scaling up and swapping slots (with preview) to have a guarateed warmup of those pages: After the configured pages have been loaded, the application will be added to the pool, or will be ready for swap (with preview)

Summary

While swapping slots is a best practice to achieve zero downtime, it does have some drawbacks, for example when working with slot specific settings. Using application initialization and the Swap with preview method will circumvent this shortcoming and deliver a true zero downtime experiene.

How to query xConnect using the tracker ContactID in Sitecore

0
0

There are situations where not all custom facets have been loaded into your session, or where you want to explicitly check for updates on a custom facet in xConnect, for example when external systems might have been making changes to this facet. This blogpost explains how to use the trackerContactID to query xconnect, which can be used to get these custom facets.

In anonymous situations, the only identifier that is available, is the current contact id:

var xdbIdentifier = Tracker.Current.Contact.ContactId;

Although it might be attached to other identities, it’s the only identifier that might be available. This ID can be used to setup a new client connection to xConnect and do a lookup for the contact, including one or more custom facets.

The IdentifiedContactReference can be used for the query, it needs to be constructed with a source (which can be any source of identification, for example twitter, facebook or Azure Active Directory) and an ID. In this case, the source is the xDB tracker and the ID is the contactID that is provided by the tracer. The source that needs to be used is “xdb.tracker”. However, the ID that the tracker provides, is a guid with dashes, while the xConnect API expects a guid without dashes. The following extension method can be used to convert this ID:

public static string ToXConnectIdentifier(this Guid contactId)
        {            
            return contactId.ToString("N");
        }

Now, the IdentifiedContactReference can be constructed and be used in your (custom) queries:

var id = new IdentifiedContactReference("xDB.Tracker", xdbIdentifier.ToXConnectIdentifier());

Happy querying!

Enabling the application map in Application Insights for Sitecore to monitor your Sitecore infrastructure and webclients

0
0

In the out of the box configuration for Sitecore on Azure, application insights has been enabled by default. However, this configuration is optimally configured. In normal situations, it’s very valuable to have insights in your infrastructure: what connections provide a lot of errors, latency or other issues. This blogpost explains on how to get these insights with for Sitecore.

It’s possible to enable Application Insights in two different ways:

  • Enable it buildtime, by adding the Application Insights SDK to the application
  • Enable it runtime, by enabling an extension on the web application

these methods will enable you to monitor your application. Whenever this has been enabled, custom logging can be written. For the serverside logging, Sitecore already has replaced the log4net logger by the application insights logger, but Sitecore has omitted the clientside logging.

Enabling the application Map by updating the buildtime configuration

Luckily, Application Insights is enabled using the SDK by default for Sitecore, but as mentioned earlier, the configuration is not optimal. The insights you would like to get, is are the following:

but all that is provided, is the following information. A single application, which displays all requests, but there isn’t any dependency information available.

This is due to the following configuration in the applicationInsights.config:

<TelemetryModules>
    <!-- Uncomment the following line to enable dependency tracking -->
    <!-- <Add Type="Microsoft.ApplicationInsights.DependencyCollector.DependencyTrackingTelemetryModule, Microsoft.AI.DependencyCollector"/> -->
    <Add Type="Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.PerformanceCollectorModule, Microsoft.AI.PerfCounterCollector">
    <! -- cut a lot if information -->
</TelemetryModules>

By removing the comment and restarting the application (this is mandatory), the detailed “web” of dependencies is shown. Every element in this overview, like the application, the different  connections and the actual services can be selected and give a different overview of information. In the case below, the sql connection has been highlighted, indicating that 19.2% of the requests cause errors:

Looking at depenencies

When taking a look at the performance blade, we now do have an overview of all the dependencies of Sitecore: xConnect, the CES discovery service, all SQL dependencies and your own custom build dependencies.

Drilling down on these depenendencies, for example, long running dependencies, might give a very insightfull overview on what is actually happening within Sitecore:

Integrate clientside logging

Adding clientside logging is easy and gives great insights as well! The first step is to include this javascript into your page:

please copy the code from this page as I can’t reference the code: it turns into a tracking pixel 😉

Using this code will result in the following information. In my case, I have a few errors that are automatically logged to Application Insights and there are some generated insights on my slowest clientside calls.

Another benefit of enabling application insights at the client, is that Microsoft is creating functionality for client analytics, like funnels, flows, et cetera. This might overlap with Sitecore analytics in some ways, but it’s always fun to have an extra bit of data available, right?

Summary

By just slighty changing the Application insights configuration, much more insights can be gained from the application map. A very interesting insight is the failed SQL statements that are triggered from your environment; this is often an indication in problems with your “tasks”  database. Happy monitoring!

 

 

Using Application Insights Annotations and how to trigger them within your application

0
0

Recently I discovered the possibility of adding notes to specific point s in time on the application insights classic metrics. This is a very interesting way of marking specific, important events in your application lifecycle, for example, deployments. It turns out, that an Azure DevOps extension exists to mark application deployments, This might be a point in time where your applications starts to behave differently, due to a bug or new functionality. These events may arise from your application as well. An interesting event could be the time of publishing content in a Content Management System, as it might influence the behaviour of your web application. This blogpost explains how to use annotations in application insights and the things that don’t work (yet)

Source code can be found here

What are annotations and how do they work?

To start with the drawbacks: Annotations only work in the classic metric explorer; they don’t show up in the new metrics yet. These annotations are small icons in the timeline, containing a certain message and specific icon.

From the UI, those annotations can be made:

and this button gives the possibility to specify a date and time, a name, a comment and a label:

From this list, one specific annotation type is missing: The deployment annotation. Deployment annotations are indicated by the small arrows in the first image of this blogpost.

Using annotations in the Failure and Performance overview

In the “new” Application Insights, these deployment annotations can be found in the failure and performance overview, which means that Microsoft marks them as important events in the application lifecycle:

 

All the other events, which can be added manually from the UI, don’t show up in this overview, and as far as I know, no filters can be applied to add other markers as well. In our specific situation, we had to need to add markers in this timeline. We are working with the content management system “Sitecore”, which has different roles, each being deployed to different application services. Certain events, such as publishing content, might have effect on the performance of the content delivery role (the web application that serves content to visitors); that’s why these events needed to be marked. As the only visible annotation was the deployment annotation and that one was not useable from the UI we had to create some code to create this annotation.

How to create annotations within your application

I couldn’t find any information about the API, but luckily, Microsoft created some powershell for this two years ago. All I had to do was to convert this code into C#. In my code repository I created a class “Annotations” which does the heavy lifting.

From the application, everything that needs to be done is executing the following code:

  var annotation = new Annotations();
  annotation.CreateAnnotation("Published content", AICategory.Deployment);

In this specific example, a deployment annotation with the title “Published content” will be made, as shown in the previous image.

Summary

Altough annotations don’t work in the new metric overview and only deployment annotations are shown in the new application insights Failure and Performance overview, it can be of great use to mark (very) important events in your application lifecycle. This API helps to do the heavy lifiting and add annotations in a very convenient way.

Using Application Insights annotations to mark publishing events in Sitecore

0
0

In my previous blogpost I described how to create annotations within an application. This blogpost explains how this was integrated in Sitecore and how it could have helped in analyzing our severe performance problems

source can be found here

Last week, we were facing severe performance issues on our content delivery server on Azure. The CPU was running 100% on certain times, and loadtimes of pages were running up to 10 minutes. The strange part was that this behaviour was quite random (at first). Some moments, the pages were served blazingly fast and other moments it might have been smarter to print the pages, put them into an envlope, and personally deliver them to the person who requested them. By feet.

Our average and 99th percentile overview looked as follows:

It turned out that the performance hits only appeared during business hours. To be specific: the performance problems started to appear around 8:30am and disappeared around 4pm:

What happened?

The TLDR; It turned out that when the Url Rewrite cache is empty, all requests will try to rebuild that cache. This is done using a search query (We use Azure search to be clear), which might throttle if too many clients are trying to use search. In the image below it is clear that often more than 30% of all search queries were throttled.

This lead to longer waiting times and even a lot of redis/session exceptions, in other words: a chain reaction of events happened which caused very strange behaviour.

The reason behind the empty cache

What was the reason behind the empty cache? THese caches can be flushed manually, or they are emptied after a content publish And that is exactly what happened throughout the day. Every few minutes, some new content was created and published, which lead to those empty caches, which caused all the requests to try to build up that cache, which lead to “fights” amongst resources. This was for me the reason to add publish annotations to the timeline; everytime a content publish will take place, this will be added to the timeline. When mapping the actual publish actions on the performance timeline, it becomes visible that the publish actions was the reason of the bad performance:

 

The implementation

The implementation was fairly simple; First, I created the annotation library (as described in my previous blogpost), the second step was to create a publishing processor which gets executed just before the actual publish action:

namespace PublishAnnotations.Pipelines.Publishing
{
    public class AnnotatePublishAction : PublishProcessor
    {        
        public override void Process(PublishContext context)
        {
            var client = new TelemetryClient(TelemetryConfiguration.Active);

            var dependency = new DependencyTelemetry();
            dependency.Name = "Annotate publish action";
            dependency.Target = "Application Insights";
            dependency.Type = "Http";
            var operation = client.StartOperation(dependency);

            try
            {
                this.AnnotatePublishingAction();
                dependency.ResultCode = 200.ToString();
                dependency.Success = true;

            }
            catch (Exception e)
            {
                dependency.Success = false;
                client.TrackException(e);
            }
            finally
            {
                client.StopOperation(operation);
            }          
        }

        public void AnnotatePublishingAction()
        {
            var annotation = new Annotations.Annotations();
            annotation.CreateAnnotation("Published content", AICategory.Deployment);
        }
    }
}

Summary

Publishing actions might have severe implications when there are components in your code which cannot handle these (regular) publishing events too well. That’s why we decided to add these annotations to the pipeline, they might have helped us in finding the issue much much faster. In the future, we will loadtest our applications including publication actions and cache flushes, to be sure that these actions will not influence the performance anymore as well.

JSS beginner issues: Placeholder ‘xxx’ was not found in the current rendering data

0
0
Currently, I am researching JSS and I must say: it’s great. So far, I ran into a few issues and although the documentation is great (I would recommend everyone to checkout the styleguide in the default app!), I am sure that people will run into the same issues as I did. I’ll share short blogpost on these issues. Today number 1:
‘Placeholder ‘xxx’ was not found in the current rendering data’
This issue occurs when component is inserted into a non-existent placeholder. This probably occurs under the following conditions: a) a new component has been created with a new placeholder
<div class="row">    
    <div class="col-sm-8 col-lg-10">
        <sc-placeholder name="jss-main-carfunnel" [rendering]="rendering"></sc-placeholder>
    </div>
    <div class="col-sm-4 col-lg-2">
      <sc-placeholder name="jss-side" [rendering]="rendering"> </sc-placeholder>
  </div>
</div>
b) the route data was updated and imported into Sitecore
id: carfunnel-step-1
fields:
  pageTitle: Carfunnel - step 1
placeholders:
  jss-main:
  - componentName: CarfunnelLayout
    placeholders:
      jss-main-carfunnel:
      - componentName: CallMeBack
        fields: 
          heading: Call Me Back. Please! Now!
The most probable cause that causes this issue, is that your component definition was not updated:
export default function(manifest: Manifest) {
  manifest.addComponent({
    name: 'CarfunnelLayout',
    icon: SitecoreIcon.Layout
  });
}
Make sure to add the placeholders to the sitecore definition: this way, Sitecore knows a placeholders exists over there:
export default function(manifest: Manifest) {
  manifest.addComponent({
    name: 'CarfunnelLayout',
    icon: SitecoreIcon.Layout,
    placeholders: ['jss-main-carfunnel', 'jss-side']
  });
}
Never forget to update your definitions. When working with fields, the error message is quite meaningful, but when working with new placeholders, these might be forgotten!

JSS beginner issues: using the ReactiveFormsModule – Can’t bind to ‘formGroup’ since it isn’t a know property of ‘form’

0
0
although being a total JSS and angular newbie, I wanted to create some nice forms, use an API and use some state in my application. The internet is full of solutions, but none of them fixed my issue. Read more after the break. Just to have a quickstart, the default angular solution, provided by the JSS framework, was used. When using the ReactiveFormsModule, most solutions suggest that the ReactiveFormsModule should be added to your @NgModule. And while they are totally right, most of them suggest that it should be added to the App Module, which did not solve the issue in this case. After some thorough research it appeared that a SharedModule was added, directly under the “components” directory:
When using the default template, all required modules can be added to this shared module. It’s even described in the contents of this file!
Because it is at the end of the complete component list, the file may be overlooked and you could end in a loooong journey to solve this n00b issue 😉 Happy coding!

Sitecore on Azure – design considerations to be more cost efficient and have more performance

0
0
After working for quite a while with a lot of Sitecore workloads on Azure, we have built up quite some experience with regards to scale and costs management. Although Sitecore has some predefined topologies, there may be various reasons why they will work or won’t work for you. From what we have seem, those topologies are not the most costs effective ones and having different requirements might lead to different choices in terms of what tier is right for you. This series of blogposts gives an overview of choices that could be made and a small indication of the costs estimation for two of the Sitecore of Azure workloads (the Single and Large setup). Please note that some choices might only be valuable for XP or only for XM, or even not be beneficial at all, as there is not cookie-cutter solution for everything.

An example

In the tiers that Sitecore defined, there is guidance to run the content management server on a B3 tier, which would cost you around EUR 112,05 a month. When the client has requirements on blue/green deployments for the CM and/or has a need for daily backups, the choice for the S3-tier could be made, as it supports out of the box staging slots and daily backups. While it costs EUR 40,- per month more, I am pretty confident that creating your own blue/green deployment strategy and creating your own backup strategy will cost you far more than these EUR 40,- per month. In this case, the choice for a slightly more expensive setup could be made.

In this blogpost some design considerations will be described that you could make to choose for a different setup as apposed to Sitecore’s guidance. This might be a choice from a developer’s perspective as well the choice from an infrastructural perspective. Please note that I only included some considerations and all considerations that I have described, all benefit from a costs- and/or performance angle.

Topics: Please note: in the end: running a website does cost money and running a large website costs more money. Increased load leads to increased CPU cycles, extra load on databases, azure search, redis and even application insights. While it is a common habit on premise to “just add a bit of memory, CPU and storage”, the most interesting part in Azure is “how can my design the most costs efficient without affecting the user experience”. This series of blogposts doesn’t have the silver bullet on “what setup does fit for everyone”, but I hope it will help with making the right choices, to have the most optimal Sitecore on Azure setup

A rough estimation of the costs following the Sitecore recommendations

In the table below there is an rough costs estimation of Sitecore on Azure within the Western Europe region. Costs may differ over regions and may differ based on your agreements with Microsoft. For example: when having an enterprise agreement, you could run you non-production workloads in Dev/Test subscriptions, which greatly reduces costs for several pricing tiers in Azure App Service plans. It gives up to a 55% reduction on SQL and let’s you run a S2 app service workload for about 60% of the costs of a production subscription. In each blogpost I will make a reference on how to possibly reduce costs. The table below shows the estimated costs following the Sitecore recommendations.
estimated costs for the XP0 setup and a large scaled XP setup
estimated costs for the XM0 setup and a large scaled XM setup
The estimated costs for a development PaaS environment is roughly EUR 726,-, while a production instance might costs around 4426,82 per month. For the XM workload, this is a lot less; EUR 488,- for a development environment versus 2011,- for a large scaled XM production workload.

The largest difference is in the database area; for the XP workload, there is a major cut on the budget due to the P1 workloads for the xConnect databases, while these databases get omitted by XM workload. It’s a roughly 25% vs a 6% part of the costs, while there still is a large dependency on search services and app services. The good news: costs for all environments can (greatly) be reduced based on smart choices!

How to: Create a DTU consumption overview using Azure Metrics

0
0
In a previous blogpost I showed a small overview of DTU consumptions of all Sitecore databases and how to use that overview to reduce your costs. This blogpost will explain step by step how to create that overview. An example of the file can be found here

move to your resource group, select metrics and add a new metric

Select all databases and select the DTU used metric

this is a bit inconvenient to do, but hey investing 5 minutes to save hundreds, maybe thousands of euro’s is worth some time, right? Make sure to select the “DTU used” metric, as it gives an absolute overview.
Make sure to select “Max” under aggregation the Max DTU consumption per interval is important in this overview, otherwise, the figures will give a wrong insight
This will lead to the following graph – make sure to select the last 30 days, to gain insights over a long period of time
a graphical overview of the total DTU consumption
Download the data as an excel workbook – this will give you the possibilities to get extra insights on the actual consumption
Download the data to Excel

Open the Excel workbook and insert three blank rows under the database row – These rows will be used to roll up all important information
Add “DTU, max DTU per DB and percentage” labels in the first column of each row
Add the DTU setting for each database on the DTU row Compute the Max usage per database – inserting the formula =MAX(B15:B1000) should be sufficient for 30 days

Insert the percentage in the percentage column – this will give an easy overview over the over/undercommitment for each database. Don’t forget to change the data format to percentage
select the max DTU and percentage formula row and pull to the right- by pulling the black cross to the right, the formula will be duplicated for every database
Compute the SUM per row – this will give an overview of the committed resources and consumed resources per hour. Pull down the formula to the end of the worksheet.

Get the max value of each SUM(row) –This will show the maximum resources which were consumed simultaneously

This will lead to the following overview. – Conditional formatting could optionally be added to give an easier overview:

To Elastic pool or not to elastic Pool for Sitecore on Azure

0
0
In the Sitecore #azure slack channel there are often discussions about pricing, scaling and performance. A common best practice which is shared in this channel is the use of Elastic Pools for your SQL databases. In this article I our findings, how you can compute the costs for elastic pools and how it will affect performance, as opposed to the default Sitecore ARM templates.

PS: the costs I share are all retrieved from the Azure pricing calculator and are not actual prices from my company – I will never share these.

This blogpost is a part of these blogpost series

Total costs of Sitecore databases on Azure Paas

The exact amount of costs depends on the way how the Sitecore environment has been scaled. In default Sitecore tiers different combinations of isolated SQL databases have been used, which result in the following Pay-as-you-go costs. At the moment that there will be a requirement on “automatic failover, geo replication” or whatsoever, the default setup does not meet these requirements and will effectively lead to a double in SQL costs, which wouldn’t be a major problem for the XM setup, but will lead to a significantly increase of costs for the XP setup.

Below are the costs for the XP/XM single and large workloads, following the Sitecore recommendations
tier Single XP Large tier XP Single XM Large tier XM
Costs 198,56 1167,- 86,- 136,53

Changing from isolated databases to an elastic pool

While Sitecore’s default setup offers isolated databases, a lot of people move over to the elastic pool model. It is often seen that the core, master or web databases consume 100% of all the computation power (especially under load, after deploying or after publishing), which has a negative performance and uptime impact on the complete environment. The elastic pool offers the possibility of reserving DTUs, which will be divided runtime between all databases. And while a master/core database will require more performance after a deployment, a web database will need more performance when under heavy load. In the isolated setup, the DTU’s that are not consumed by all of the databases cannot be used by the database that requires it at that very moment, which is possible with the elastic pool model.

An analysis of a workload running in production

In this overview I’ll show a roll up of the DTU consumption of databases during the last week of last year and first week of this year on an environment that had quite some load. Please not that we didn’t use forms and that we didn’t enable the analytics on the Content Delivery servers, so this might give a distorted view in comparison withyour environment, but it helped us to scale our environment delivery to save a serious amount of money:
rollup of database resource consumption for a production workload – Learn how to create your own overview here
Legend:
  1. The total amount of committed DTU
  2. The sum of the maximum use of DTU of each database at any given time in these two weeks
  3. The max DTU usage of the database in these two weeks
  4. Maximum percentage of assigned DTU that was consumed in these two weeks
  5. The maximum hourly total DTU consumption
This simple overview gives a lot of valuable insights:
  • If every database was running max effort simultaneously, that we would have consumed just 43% of the reserved capacity – which is a waste of money
  • In this actual view, we were just consuming 82 DTU at most during these two weeks: just 24% of all resources have been consumed. This is even a larger waste of money
  • Some databases didn’t do anything: marketing automation, forms, exm, smm, tasks – a waste of money. The DTUs were committed but couldn’t be used by other databases
  • The master and core databases didn’t cut it at certain times. This happened after deployments and after publishing. It would have been great if the tasks, forms and marketing automation resources could have been used during those times

A solution: Move databases to the elastic pool

While there was an overcommitment, some databases still didn’t cut it. Using an elastic pool is an excellent solution for these problems. As we were just consuming 82 DTU at max, the move to a 100DTU elastic pool is quite interesting: while the total costs for a single instance isolated database setup are EUR 199,-, the large production workload costed us around EUR 430,- (we didn’t scale shard0 and shard1 to P1, as we didn’t use the xconnect services too much yet).

By moving to a 100 DTU elastic pool, the costs could have been reduced to 186,-, according to the azure pricing calculator.

The downside of a single elastic pool

What every advantage, a disadvantage shows up. With every database in a single pool, a situation could occur where the xconnect databases and the web databases are fighting for resources. As the resources are not isolated anymore, this could lead to decreased performance. It might be an option to assign different sets of databases to different pools, for example.
  • Web database (isolated database)
  • All other Sitecore databases (core, master, forms) in a single pool
  • All xConnect related databases in a different pool

Changing the purchase model from DTU to vCore

When diving into the Azure SQL pricing options, it becomes clear that a price reduction on this setup is not possible, however, a change to use the vCore model, instead of a DTU model, opens up the possibility to make use of the Azure hybrid benefit (bring your own on premise sql license) and the use of reserved resources; resources could be reserved for the next 3 years. A combination of both options could save up to 79% of costs! (more on that later)

Differences between DTU and vCore

The DTU-based model basically is a linear combination between computing power and storage, whereas the vCore model delivers a much more flexible way of assigning compute-resources:
DTU model vs the vCore model
The general rule of thumb: Each 100 DTU in the standard tier requires at least 1 vCore in the general purpose tier. More info on these model can be found here.

When applying this model on the information that we just have analysed, a choice could be made for a 1vCore general purpose setup. When bringing own licenses and reserving resources for the next three years, it could bring the price down to just 46,- in a Gen 4 setup or 92,- for 2 vCores in a Gen 5 setup. Looking back at the estimated costs, that is a reduction for every workload!
Gen 4 vCore pricing options
Gen 5 vCore pricing options
In the end the costs for the production workload have been reduced from the initial 430,- (in our case) to almost a tenth of this amount; 46,-.

Summary

The Sitecore default recommendations leave room for performance and costs effective choices. In our case, creating a simple overview of the committed DTU’s and the actual usage lead to a enormous decrease in costs, while having better performance. Every situation is of course different. This setup works for us, but might not work for you. The numbers tell the tales: measure, gain insights, get your conclusions and see what optimizations from a performance and a resource consumption view could be applied in your case. Make sure to gain insights in what databases are (over)commited, what databases are undercommited, what databases could share an elastic pool and create a new design for that situation.

Azure Search for Sitecore design considerations

0
0
Azure Search makes up for a significant amount of costs within a Sitecore on Azure setup; apart from these costs, there are often issues with Azure Search and clients tend to move away from Azure Search and spin up a new VM with Solr – But is this really mandatory? this blogpost is part of a series of blogposts

Sitecore developers are spoiled

Most Sitecore developers are very spoiled; on premise hardware is virtually unlimited. For example: when working with Solr, there is no need to take care of any limitations – but things change when the workload moves to Azure PaaS, as Azure Search does have some limitiations. But how severe are these limitations?

Azure Search limits

When looking at the Azure Search limits there are a few important ones to take care of:
  Free Basic S1 S2 S3 S3 HD L1 + L2
Indexes 3 15 50 200 200 3000 10
Fields 1000 100 1000 1000 1000 1000 1000
Scale units 0 3 36 36 36 36 36
Costs 0 62.18 206.84 827.38 1654.76 1181.35 2363.32

Indexes

Azure Search has an maximum index limit – while Sitecore comes with 13 indexes out of the box (including the xdb and xdb_rebuild index), this leaves very few possiblities open for custom indexes. This also means that the Free tier and L1 + L2 tiers cannot be used by Sitecore, as they don’t support enough indexes. Another consequence is that the basic and S1 tiers cannot host multiple Sitecore environments, if you would share an Azure Search service over multiple instances.

Fields

Every tier, except the basic tier, supports 1000 fields. This 1000-field limit is strict and could lead to severe problems when not being monitored correctly. Due to the 100 field limit on the basic tier, this basic tier can not be used without any modifications to Sitecore (more on that later)

Scale units

Scale units are the amount of replicas multiplied by the amount of partitions. The default settings are “1”, but when more concurrent searches or faster queries are needed, a replica or partition could be added. To explain how this works, the “telephone book analogy” could be used:

telephone book analogy

Imagine a single telephone book with 1000 telephone numbers. Just one person could use that telephone book to lookup a certain telephone number. To be able to have multiple persons search through that index, a replica of that telephone book should be created. For every replica, an extra user can execute a concurrent search through the index.

While the index with telephone numbers grows, it might take longer and longer to do a proper lookup for a telephone number. An increase to 10000 telephone numbers in the index might lead to a performance degradation. In this case, one more more partitions could be added: the index would be split into a partitioned phonebook with last names starting with A-M and
a partion with lastnames starting with N-Z. To be able to lookup a telephone number with multiple concurrent users, each partition would need its own replica. In the end you would and up with an ‘X’ amount of partitions, each with ‘Y’ replicas, which would lead to a total of X*Y books: the amount of Scale units in terms of search.

Getting the most out of Azure Search for Sitecore

Getting the most out of Azure Search isn’t too hard, the only thing is that you might to unlearn some old habits. In the end it might lead to a bit more configuration, while it might save tons of money
  • Work around the 1000 field-limit
    • Index only what you need
    • Create custom indexes
  • Cache, Cache, Cache
  • Scale your Search service (or don’t)
  • Consider an external Search solution

Work around the 1000 field-limit (aka the friends with benefits method)

Most Sitecore developers have a background with Solr, which doesn’t have too many limitations. It could virtually host an unlimited amount of fields, which is not the case with Azure Search. All tiers, except the basic tier, can hold a maximum of 1000 fields, due to the way Azure Search copes with it’s memory. This limit can be reached really fast, when working with small, autonomous templates, SXA, and/or multiple languages. This is due to the fact that Sitecore has a default setting on its master and web indexes named “indexAllFields=’true'”.

Set ‘indexAllFields to ‘false’

This will prevent all fields from being indexed. Sitecore does come with a configuration in which all required fields for the content editor view will work (title, body, buckets and some other computed fields) – thus this will greatly reduce the amount of fields.

The drawback, however, is that your complete search functionality will break, as no custom field is in the index anymore.

Create a custom index for ‘web’ and ‘master’ and put your custom fields in these indexes

Using this pattern, will keep the out of the box index very clean and fast. Sitecore could add whatever field they want to this clean index, without affecting your indexes. Add all fields that are used within your custom search functionality into the custom indexes. This will lead initially to a bit of custom configuration, but it will lead to very small and manageable indexes as well.

Extra free benefits – faster indexing actions and faster queries

Faster indexing actions
The extra benefits in this approach are great: because just a very small subset of all data is included, the (re)-index of the Sitecore content decreases from several minutes to several seconds. This helps in the blue green deployments and uptime of your environment. the search queries are faster and smaller
When querying Azure search, all fields in the index are returned – even if they are empty. When having 900+ fields, this will lead to very large responses (900 fields per result) and long response times (as Azure Search searches through all fields).

After the optimization of having a custom index with only the fields that are really needed, this will mean that for each result only a few fields are returned and because of the small dataset, the query is much faster than before. Move to Plan B (the pricing tier) – More and faster queries for lower costs Due to this approach, its very likely that your sitecore indexes would stay below the 100 fields. This means that the the “Azure Search S1′ level could be reduced to the “Azure Search tier B” level. This could lead to a costs reduction of 206-64 = 142 EUR per environment, just by using a different configuration. Especially for your dev, test and QA environments.

Note: I have not researched the impact of xConnect for this approach
Note2: Tier ‘B has a maximum of 3 Scale units. If you need more query power, you should still consider the S1 tier. But this is often not applicable for dev and test environments.

Cache, Cache, Cache

Although search is a very fast pattern to lookup data, it still takes time. When your result set is always the same (for example, when looking up a certain set of articles of a specific category), this dataset should be cached and not be executed on every request. With an increasing load on your webserver, this would lead to an increasing load on your search service, which could mean that an increase in replicas would be needed. When having just one partition, this could be,- EUR 64 (tier B), EUR 206 (S1) or EUR 827,- (S2), but when working with multiple partitions, this could lead to a significant increase of costs.

An easy approach is to just turn on your output cache, as you would normally do on your Sitecore renderings, other approaches might involve custom Sitecore caches. The key in here is: be as cheap as possible

Scale your Search

After these changes, it might be possible that you could scale down your search services. Make sure to execute some performance tests on your QA environment before you scale down, or you might end up in error with your production environment. Scale down could mean: scale down to a lower tier or reduce the number of replicas and partitions.

Consider an external Search solution

Yes. Although developer tend to solve everything with custom code and own frameworks, a site that heavily depends on search might benefit from external search solution such as Coveo. They offer a lot of functionalities to the content editors out of the box for a price that no developer could build it themselves. 2000,- a month sounds a lot, but this is barely 20 hours of development for a single developer: I bet that in those 20 hours no deployments, bug fixes, new functionality and a fully scalable solution could be build.

moving to an external search solution would mean that that vendor takes care of the performance for the frontend searches and you would only have to take care about the performance on content management and xConnect environments.

Indications that you might need to redesign your search approach

The best indicator is the Azure Search Metrics. In this overview, you can display graphs on query latency, query throttling and the amount of queries. When there is any sign of query throttling, this means that these queries are not executed and thus, leads to problems in your portal.
Query throttling kicks in
The issue in the picture above happened due to a misconfiguration, all other days show correct behaviour. The graph below gives more food for thought. The red lines are search queries, while the purple lines are throttled queries. An interesting fact: during the throttling almost no queries were possible: something really messed that search service up during that time!

Gotchas

Moving up or down a service tier is not possible

When deciding to go with a “Tier B” search, keep in mind that upgrading to a “Tier SX” is not possible. You would require to delete your Search service, and recreate it, to get a service with the same name. Getting the same apikeys is not possible.

SXA

SXA adds all the template fields to the index by default. By turning this off, you would have to manually add all the fields. While SXA would only need actions within the Sitecore environment and no configuration changes, this strategy might not be a suitable solution for you.

Summary

Azure search costs can “rise out of the pan” (this is a dutch saying) increase very fast and lead to errors pretty quick, when not correctly scaled. By being “cheap on resources”, these issues could easily be overcome, which would still allow you to run your Sitecore on Azure completely as a PaaS setup, instead of having to add an IaaS based Solr instance.
Viewing all 54 articles
Browse latest View live




Latest Images