Service Fabric and Dependency Injection, part 3: Unit testing

This is a series of blog posts:

Unit testing of the final construct we’ve got in previous two parts would be challenging due to a number of reasons:

  • Both StatelessServiceContext and StatefulServiceContext are sealed, what means you cannot instantiate them directly, ContainerConfig(new StatefulServiceContext()) simply won’t compile, neither you can easily inherit one or another, e.g. class MockedStatelessServiceContext : StatelessServiceContext.
  • Both StatelessService and StatefulService require the exact type of their respective context type, i.e. public StatelessService(StatelessServiceContext context), what means you cannot mock their base class so Mock.Of() won’t satisfy service’s ctor and it will fail in runtime (or in test time if you have appropriate test, and you should!).

Would be a bleak picture which would bury the whole idea. But ServiceFabric.Mocks to the rescue! Thank to it we can easily mock both contexts:

[TestClass]
public class ContainerConfigTest
{
    [TestMethod]
    public void Verify_MyStatefulService_Container_Should_Not_Throw_Exception()
    {
        // Arrange
        var container = ContainerConfig.CreateContainer(MockStatefulServiceContextFactory.Default);

        // Act
        // Assert
        container.Verify(VerificationOption.VerifyAndDiagnose);
    }

    [TestMethod]
    public void Verify_MyStatelessService_Container_Should_Not_Throw_Exception()
    {
        // Arrange
        var container = ContainerConfig.CreateContainer(MockStatelessServiceContextFactory.Default);

        // Act
        // Assert
        container.Verify(VerificationOption.VerifyAndDiagnose);
    }
}

That’s it! Now both of your stateless and statefull services themselves and all their dependencies are recursively instantiated and controlled by the container, plus their registrations are tested.

Posted in Programming | Tagged , | Leave a comment

Service Fabric and Dependency Injection, part 2: Stateless Service

This is a series of blog posts:

Last time we instantiated a stateful service from a DI container, it was elegant and relatively easy. What I can’t say about instantiating a stateless service. The problem is that it needs a reference to the same container in 3 different places: Program.cs, MyStatelessService.cs, Startup.cs.

First the near identical code between stateless and statefull services:

ServiceRuntime.RegisterServiceAsync(nameof(MyStatelessService) + "Type", context => CreateService(context))
              .GetAwaiter()
              .GetResult();

private static StatelessServiceBase CreateService(StatelessServiceContext context)
{
    var container = ContainerConfig.CreateContainer(context);
    return container.GetInstance<MyStatelessService>();
}

Now where the differences start. Let’s start from refactoring the Startup class. From the template it comes as a so-called conventional startup, in other words follows the Convention over Configuration principle. Now it inherits StartupBase and accepts Container in its constructor:

internal sealed class Startup : StartupBase
{
   private readonly Container _container;

   public Startup(Container container)
   {
       _container = container;
   }
}

Then inject IStartup into service’s ctor alongside its other dependencies (if any):

internal sealed class MyStatelessService : StatelessService
{
    private readonly IStartup _startup;

    public MyStatelessService (StatelessServiceContext context, IStartup startup, ...)
       : base(context)
    {
        _startup = startup;
    }

    protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
    {
        yield return new ServiceInstanceListener(serviceContext =>
            new HttpSysCommunicationListener(serviceContext, nameof(MyStatelessService) + "Endpoint", (url, listener) =>
            {
                var builder = new WebHostBuilder();
                return builder.UseHttpSys()
                              .ConfigureServices(services => services.AddSingleton(serviceContext)
                                                                     .AddSingleton(_startup))
                              .UseContentRoot(Directory.GetCurrentDirectory())
                              .UseServiceFabricIntegration(listener, ServiceFabricIntegrationOptions.None)
                              .UseUrls(url)
                              .Build();
            }));
    }
}

The CreateServiceInstanceListeners() method differs from the template just slightly. Instead of UseStartup() now we use AddSingleton(_startup). What means the Startup class should be registered in the container by its interface:

container.RegisterInstance<ServiceContext>(context);
container.RegisterInstance<StatelessServiceContext>(context);
container.RegisterSingleton<MyStatelessService>();

container.RegisterSingleton<IStartup, Startup>();

The only thing left is to configure the ASP.NET Core infrastructure pipeline to use the container, for more details see the documentation:

public override void ConfigureServices(IServiceCollection services)
{
   services.AddSingleton<IControllerActivator>(new SimpleInjectorControllerActivator(container));
   services.AddSingleton<IViewComponentActivator>(new SimpleInjectorViewComponentActivator(container));

   services.EnableSimpleInjectorCrossWiring(container);
   services.UseSimpleInjectorAspNetRequestScoping(container);
}

public override void Configure(IApplicationBuilder app)
{
    container.AutoCrossWireAspNetComponents(app);
}

That’s it! Now your statefull service itself and all its dependencies are recursively instantiated and controlled by the container.

Posted in Programming | Tagged , | Leave a comment

Service Fabric and Dependency Injection, part 1: Stateful Service

This is a series of blog posts:

The template of a Service Fabric application that is shipped with Visual Studio instantiates all services explicitly. Like and any other basic template, such as of ASP.NET infrastructure pipeline, it instantiates all filters and handlers right away as well.

But sooner or later you’ll likely have to pass to your service some dependencies, which have their own dependencies, and so on. This way you end up having a dependency tree. Natural answer to its growing complexity would be using a Dependency Injection (DI) container. I cannot recommend more Simple Injector for its performance and elegance of API.

Another reason to use a container would be the design principle which suggests that all constructor calls in your code (the new keyword in C-like languages) are the aspects of explicit lifetime (aka lifestyle) management scattered across your codebase. While actually it has to be performed by a container in one, centralized place called composition root.

ServiceRuntime.RegisterServiceAsync(nameof(MyStatefulService) + "Type", context => CreateService(context))
              .GetAwaiter()
              .GetResult();

private static StatefulServiceBase CreateService(StatefulServiceContext context)
{
    var container = ContainerConfig.CreateContainer(context);
    return container.GetInstance<MyStatefulService>();
}

Now you only need to register service (optionally) and service context (mandatory):

internal static class ContainerConfig
{
    public static Container CreateContainer(StatefulServiceContext context)
    {
        var container = new Container();
        container.Options.DefaultScopedLifestyle = new AsyncScopedLifestyle();

        container.RegisterInstance<ServiceContext>(context);
        container.RegisterInstance<StatefulServiceContext>(context);
        container.RegisterSingleton<MyStatefulService>();

        return container;
    }
}

The caveat here is to map service context to two types, StatefulServiceContext and ServiceContext. Why? Because you’ll likely have both steteful and stateless services in your application, and their dependencies, such as configuration management, will likely depend on the base type ServiceContext. While services require the concrete type in their respective constructor. The good things is that the same instance would be resolved in both cases:

public sealed class MyStatefulService
{
    public MyStatefulService(StatefulServiceContext context, ...)
    {
    }
}

public sealed class ServiceFabricConfigurationManager : IConfigurationManager
{
    public ServiceFabricConfigurationManager(ServiceContext context, ...)
    {
    }
}

That’s it! Now your stateless service itself and all its dependencies are recursively instantiated and controlled by the container.

Posted in Programming | Tagged , | Leave a comment

How to deploy Service Fabric application using ARM template and PowerShell, part 2: ARM

This is a series of blog posts:

To publish an ARM template using whichever mechanism you’d like, first you need the actual ARM template. Here’s an example of an application consisting of one stateless service (the front-end) and one stateful service (the back-end):

{
  "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "clusterName": {
      "type": "string",
      "metadata": {
        "description": "Name of your cluster - Between 3 and 23 characters. Letters and numbers only."
      }
    },
    "appPackageUrl": {
      "type": "string",
      "metadata": {
        "description": "The URL to the application package sfpkg file."
      }
    },
    "applicationTypeVersion": {
      "type": "string",
      "metadata": {
        "description": "The application type version."
      }
    },
    "applicationParameters": {
      "type": "object",
      "metadata": {
        "description": "Application parameters override to be applied when creating or upgrading an application."
      }
    },
    "MyStatelessService_InstanceCount": {
      "type": "string",
      "defaultValue": "",
      "metadata": {
        "description": "Dynamically generated parameter for service type MyStatelessService"
      }
    },
    "MyStatefulService_TargetReplicaSetSize": {
      "type": "string",
      "defaultValue": "",
      "metadata": {
        "description": "Dynamically generated parameter for service type MyStatefulService"
      }
    },
    "MyStatefulService_MinReplicaSetSize": {
      "type": "string",
      "defaultValue": "",
      "metadata": {
        "description": "Dynamically generated parameter for service type MyStatefulService"
      }
    },
    "MyStatefulService_PartitionCount": {
      "type": "string",
      "defaultValue": "",
      "metadata": {
        "description": "Dynamically generated parameter for service type MyStatefulService"
      }
    }
  },
  "variables": {
    "clusterLocation": "[resourcegroup().location]",
    "applicationName": "MyProject",
    "applicationTypeName": "MyProjectType"
  },
  "resources": [
    {
      "apiVersion": "2017-07-01-preview",
      "type": "Microsoft.ServiceFabric/clusters/applicationTypes",
      "name": "[concat(parameters('clusterName'), '/', variables('applicationTypeName'))]",
      "location": "[variables('clusterLocation')]",
      "dependsOn": [],
      "properties": {
        "provisioningState": "Default"
      }
    },
    {
      "apiVersion": "2017-07-01-preview",
      "type": "Microsoft.ServiceFabric/clusters/applicationTypes/versions",
      "name": "[concat(parameters('clusterName'), '/', variables('applicationTypeName'), '/', parameters('applicationTypeVersion'))]",
      "location": "[variables('clusterLocation')]",
      "dependsOn": [
        "[concat('Microsoft.ServiceFabric/clusters/', parameters('clusterName'), '/applicationTypes/', variables('applicationTypeName'))]"
      ],
      "properties": {
        "provisioningState": "Default",
        "appPackageUrl": "[parameters('appPackageUrl')]"
      }
    },
    {
      "apiVersion": "2017-07-01-preview",
      "type": "Microsoft.ServiceFabric/clusters/applications",
      "name": "[concat(parameters('clusterName'), '/', variables('applicationName'))]",
      "location": "[variables('clusterLocation')]",
      "dependsOn": [
        "[concat('Microsoft.ServiceFabric/clusters/', parameters('clusterName'), '/applicationTypes/', variables('applicationTypeName'), '/versions/', parameters('applicationTypeVersion'))]"
      ],
      "properties": {
        "provisioningState": "Default",
        "typeName": "[variables('applicationTypeName')]",
        "typeVersion": "[parameters('applicationTypeVersion')]",
        "parameters": "[parameters('applicationParameters')]",
        "upgradePolicy": {
          "upgradeReplicaSetCheckTimeout": "01:00:00.0",
          "forceRestart": "true",
          "rollingUpgradeMonitoringPolicy": {
            "healthCheckWaitDuration": "00:02:00.0",
            "healthCheckStableDuration": "00:05:00.0",
            "healthCheckRetryTimeout": "00:10:00.0",
            "upgradeTimeout": "01:00:00.0",
            "upgradeDomainTimeout": "00:20:00.0"
          },
          "applicationHealthPolicy": {
            "considerWarningAsError": "true",
            "maxPercentUnhealthyDeployedApplications": "0",
            "defaultServiceTypeHealthPolicy": {
              "maxPercentUnhealthyServices": "0",
              "maxPercentUnhealthyPartitionsPerService": "0",
              "maxPercentUnhealthyReplicasPerPartition": "0"
            }
          }
        }
      }
    },
    {
      "apiVersion": "2017-07-01-preview",
      "type": "Microsoft.ServiceFabric/clusters/applications/services",
      "name": "[concat(parameters('clusterName'), '/', variables('applicationName'), '/', 'MyProject~MyStatelessService')]",
      "location": "[variables('clusterLocation')]",
      "dependsOn": [
        "[concat('Microsoft.ServiceFabric/clusters/', parameters('clusterName'), '/applications/', variables('applicationName'))]"
      ],
      "properties": {
        "provisioningState": "Default",
        "serviceKind": "Stateless",
        "correlationScheme": [],
        "serviceLoadMetrics": [],
        "servicePlacementPolicies": [],
        "serviceTypeName": "MyStatelessServiceType",
        "placementConstraints": "(NodeType != sysnode)",
        "instanceCount": "[parameters('MyStatelessService_InstanceCount')]",
        "partitionDescription": {
          "partitionScheme": "Singleton"
        }
      }
    },
    {
      "apiVersion": "2017-07-01-preview",
      "type": "Microsoft.ServiceFabric/clusters/applications/services",
      "name": "[concat(parameters('clusterName'), '/', variables('applicationName'), '/', 'MyProject~MyStatefulService')]",
      "location": "[variables('clusterLocation')]",
      "dependsOn": [
        "[concat('Microsoft.ServiceFabric/clusters/', parameters('clusterName'), '/applications/', variables('applicationName'))]"
      ],
      "properties": {
        "provisioningState": "Default",
        "serviceKind": "Stateful",
        "correlationScheme": [],
        "serviceLoadMetrics": [],
        "servicePlacementPolicies": [],
        "serviceTypeName": "MyStatefulServiceType",
        "placementConstraints": "(NodeType != sysnode)",
        "hasPersistedState": "true",
        "defaultMoveCost": "Zero",
        "replicaRestartWaitDuration": "00:01:00.0",
        "quorumLossWaitDuration": "00:02:00.0",
        "standByReplicaKeepDuration": "00:00:30.0",
        "targetReplicaSetSize": "[parameters('MyStatefulService_TargetReplicaSetSize')]",
        "minReplicaSetSize": "[parameters('MyStatefulService_MinReplicaSetSize')]",
        "partitionDescription": {
          "partitionScheme": "UniformInt64Range",
          "count": "[parameters('MyStatefulService_PartitionCount')]",
          "lowKey": "-9223372036854775808",
          "highKey": "9223372036854775807"
        }
      }
    }
  ]
}

Happy deployment!

Posted in Programming | Tagged , , | Leave a comment

How to package Service Fabric application into SFPKG in .NET Core using MSBuild task

This task wouldn’t require much efforts but:

Here’s the full project, I called it package.props and imported from all my sfproj:

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="15.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <Import Project="..\packages\RoslynCodeTaskFactory.2.0.7\build\RoslynCodeTaskFactory.props" />
  <UsingTask TaskName="Zip"
             TaskFactory="CodeTaskFactory"
             AssemblyFile="$(RoslynCodeTaskFactory)"
             Condition="'$(RoslynCodeTaskFactory)' != ''">
    <ParameterGroup>
      <SourceDirectoryName ParameterType="System.String" Required="true" />
      <DestinationArchiveFileName ParameterType="System.String" Required="true" />
    </ParameterGroup>
    <Task>
      <Reference Include="System.IO.Compression.FileSystem" />
      <Using Namespace="System.IO.Compression" />
      <Code Type="Fragment" Language="cs">
      <![CDATA[
        try
        {
          if (File.Exists(DestinationArchiveFileName))
          {
            Log.LogMessage("File {0} already exists, deleting.", DestinationArchiveFileName);
            File.Delete(DestinationArchiveFileName);
          }
          string directoryName = Path.GetDirectoryName(DestinationArchiveFileName);
          if (!Directory.Exists(directoryName))
          {
            Log.LogMessage("Directory {0} doesn't exist, creating..", directoryName);
            Directory.CreateDirectory(directoryName);
          }
          Log.LogMessage("Zipping directory {0} to {1}", SourceDirectoryName, DestinationArchiveFileName);
          ZipFile.CreateFromDirectory(SourceDirectoryName, DestinationArchiveFileName);
          return true;
        }
        catch(Exception ex)
        {
          Log.LogErrorFromException(ex);
          return false;
        }
      ]]>
      </Code>
    </Task>
  </UsingTask>
  <Target Name="Package" DependsOnTargets="$(PackageDependsOn)" AfterTargets="Build" Returns="@(_AllPackageFiles)" />
  <Target Name="ZipPackage" AfterTargets="Package">
    <PropertyGroup>
      <PackageSource>pkg\$(Configuration)</PackageSource>
      <PackageDestination>sfpkg\$(MSBuildProjectName).sfpkg</PackageDestination>
    </PropertyGroup>
    <Zip SourceDirectoryName="$(PackageSource)" DestinationArchiveFileName="$(PackageDestination)" />
  </Target>
</Project>

It will zip folder pkg\Release to sfpkg\.sfpkg.

Now call the Package target using PowerShell:

param
(
[Parameter(Mandatory=$true)]
[ValidateSet('Debug', 'Release')]
[string]$Configuration
)

$projects = Get-ChildItem *.sfproj -Recurse
foreach ($project in $projects)
{
  Write-Host "Packaging project $project and configuration $Configuration"
  & msbuild "$project" /t:Package /p:Platform=x64 /p:Configuration=$Configuration /m
}

That’s it. Now you can take the resulting sfpkg and deploy it directly to your Service Fabric cluster.

Posted in Programming | Tagged , , | Leave a comment

How to deploy Service Fabric application using ARM template and PowerShell, part 1: PowerShell

This is a series of blog posts:

  • Part 1: PowerShell
  • Part 2: ARM

Prerequisites:

  • You have your cluster up and running. How to achieve that is outside of the scope of this series
  • Cluster has all necessary certificates deployedddeplo as for SSL endpoints, if any)
  • You have ARM template and parameters files ready. How to achieve that see the Part 2 of this series
  • You have the necessary permissions for deployment of new resources, namely the Contributor role (on the level of subscription or resource group)
  • You have destination resource group created

You need to execute the following commands:

  1. Copy-ServiceFabricApplicationPackage
  2. Register-ServiceFabricApplicationType
  3. New-AzureRmResouceGroupDeployment

Happy deployment!

Posted in Programming | Tagged , , | Leave a comment

How to remove ARM resources for Service Fabric application and service using PowerShell

First you need to remove the resource corresponding to the application itself:

Remove-AzureRmResource
-ResourceId /subscriptions/$subscriptionId/resourcegroups/$resourceGroupName/providers/Microsoft.ServiceFabric/clusters/$clusterName/applications/$applicationName
-ApiVersion "2017-07-01-preview"
-Force
-Verbose

And then the resource corresponding to the service:

Remove-AzureRmResource
-ResourceId /subscriptions/$subscriptionId/resourcegroups/$resourceGroupName/providers/Microsoft.ServiceFabric/clusters/$clusterName/applicationTypes/$serviceTypeName/versions/$serviceTypeVersion
-ApiVersion "2017-07-01-preview"
-Force
-Verbose

Posted in Programming | Tagged , , | Leave a comment

How to call LINQ extension methods such as Where and ToDictionary using PowerShell

If you have IEnumerable<XElement> and want to call various LINQ extension methods such as Enumerable.Where() or ​​Enumerable.ToDictionary() on it using PowerShell then first load the necessary assemblies:

[void][System.Reflection.Assembly]::LoadWithPartialName('System.Core')
[void][System.Reflection.Assembly]::LoadWithPartialName("System.Xml.Linq")

And here’s the code for Where():

$whereCond = [System.Func``2[System.Xml.Linq.XElement, bool]] { $args[0].Attribute("Name").Value.Contains('_') -eq $false };
$elems = [System.Linq.Enumerable]::ToDictionary($elems, $whereCond)

and for ToDctionary():

$keySelector = [System.Func``2[System.Xml.Linq.XElement, string]] { $args[0].Attribute("Name").Value }
$elementSelector = [System.Func``2[System.Xml.Linq.XElement, string]] { $args[0].Attribute("Value").Value }
$dic = [System.Linq.Enumerable]::ToDictionary($elems, $keySelector, $elementSelector)
Posted in Programming | Tagged , | Leave a comment

The BuildLayout of the application is invalid. Code is missing for service

If you’re getting these very rare but still mysterious errors:

Started executing script ‘Deploy-FabricApplication.ps1’. Test-ServiceFabricApplicationPackage : The BuildLayout of the application in
c:\Temp\TestApplicationPackage_3811564851592\icw0q1rk.bus\Release is invalid. Code is missing for service MyWebApiPkg.

or

Register-ServiceFabricApplicationType : The BuildLayout of the application in
C:\SfDevCluster\Data\ImageBuilderProxy\AppType\a9f07f16-1fda-4319-a5e6-829f38a8f093 is invalid. Code is missing for service MyWebApiPkg.

when trying to debug a Service Fabric application locally or just packaging it then make sure Build Configuration Manager settings match:

service-fabric_build-configuration-manager

Any discrepancy in Platform or Build would cause the said errors.

Posted in Programming | Tagged , | Leave a comment

How to reuse lambda parameter across multiple combined expressions

If you’re having multiple separately defined expressions having common lambda parameter like these:

Expression<Func> pre1 = u => u.FirstName != null;
Expression<Func> pre2 = u => u.MiddleName != null;
Expression<Func> pre3 = u => u.LastName != null;

And trying to combine them into one and to reuse the said parameter like this:

ParameterExpression param = Expression.Parameter(typeof(User), "u");

var predicates = GetPredicates();
var body = predicates.Select(exp => exp.Body)
                     .Aggregate((left, right) => Expression.AndAlso(left, right));
Expression<Func> lambda = Expression.Lambda<Func>(body, param);
Func func = lambda.Compile();

Then you’ll likely get an exception:

Unhandled Exception: System.InvalidOperationException: variable ‘u’ of type ‘User’ referenced from scope ”, but it is not defined.

The reason is that spite the lambda parameters have same type and name they’re defined in different expressions thus in different scopes so can’t be simply reused.

What you need to do is to use single parameter across all expressions, in other words to unify/merge/replace their parameters with one:

class ParameterReplacer : ExpressionVisitor
{
  private readonly ParameterExpression _param;

  private ParameterReplacer(ParameterExpression param)
  {
    _param = param;
  }

  protected override Expression VisitParameter(ParameterExpression node)
  {
    return node.Type == _param.Type ? // if types match on both of ends
      base.VisitParameter(_param) : // replace
      node; // ignore
  }

  public static T Replace(ParameterExpression param, T exp) where T : Expression
  {
    return (T)new ParameterReplacer(param).Visit(exp);
  }
}

And then:

var body = predicates.Select(exp => exp.Body)
                     .Select(exp => ParameterReplacer.Replace(param, exp))
                     .Aggregate((left, right) => Expression.AndAlso(left, right));

var lambda = Expression.Lambda<Func>(body, param);

That’s it, now compilation will work fine and produce the desired predicate.

Happy expressions building!

Posted in Programming | Tagged , | Leave a comment