How to enable tracing in ASP.NET Web API using Application Insights

If you were wondering on other day what this line from the default template does:

    config.EnableSystemDiagnosticsTracing();

Then know that it does exactly what is says: enables tracing using system.diagnostics API.

To forward this tracing to Application Insights, you need to install these two packages:

Install-Package Microsoft.AspNet.WebApi.Tracing
Install-Package Microsoft.ApplicationInsights.TraceListener

and add a trace listener:

<system.diagnostics>
  <trace autoflush="true">
    <listeners>
      <add name="AppInsightsListener" type="Microsoft.ApplicationInsights.TraceListener.ApplicationInsightsTraceListener, Microsoft.ApplicationInsights.TraceListener" />
    </listeners>
  </trace>
</system.diagnostic

That’s it! The Trace category is now being populated:

asp.net web api tracing ai

Happy tracing!

The ObjectContext instance has been disposed and can no longer be used for operations that require a connection

TL;DR: Don’t return task before disposing context, await it instead

If you’re getting an Entity Framework exception:

System.ObjectDisposedException: The ObjectContext instance has been disposed and can no longer be used for operations that require a connection
at System.Data.Entity.Core.Objects.ObjectContext.ReleaseConnection()
at System.Data.Entity.Core.Objects.ObjectContext.d__3d`1.MoveNext()
— End of stack trace from previous location where exception was thrown —
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Data.Entity.Core.Objects.ObjectContext.d__39.MoveNext()

in the code such as:

public Task<Order> GetOrders()
{
    using (var dbContext = new MyDbContext(connectionString))
    {
        return dbContext.Orders.ToArrayAsync()
    }
}

then it means that you’re disposing the context before the task is completed. Await it, indeed:

public async Task<Order> GetOrders()
{
    using (var dbContext = new MyDbContext(connectionString))
    {
        return await dbContext.Orders.ToArrayAsync()
    }
}

Happy awaiting!

Could not load file or assembly ‘Newtonsoft.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed’ or one of its dependencies. The located assembly’s manifest definition does not match the assembly reference

TL;DR: Add reference to Newtonsoft.Json.dll in every project

If you’re getting a runtime exception:

System.IO.FileLoadException: Could not load file or assembly ‘Newtonsoft.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed’ or one of its dependencies. The located assembly’s manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040).

which by the way is impossible to troubleshoot by searching in Google or on StackOverflow because it’s way too generic and has a millions of possible causes and solutions.

In our particular case, we had to make sure that every project that directly and (what is more likely) indirectly depends on this assembly, has a reference to the version you’re using, has the nuget package installed.

Checking you have correct assembly redirects in-place won’t hurt as well:

<configuration>
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-8.0.0.0" newVersion="8.0.0.0" />
      </dependentAssembly>
  </runtime>
</configuration>

Happy building!

How to generate ids programmatically using SQL Server and Entity Framework

TL;DR: create a sequence and execute stored proc to read from it

Generating ids in the database is used to be the default and near to the only one approach but now days this trend is fading away. Here’s some scenarios discovered by me recently where it doesn’t play well:
CQRS (see the comments, also Steven’s blog)
– bulk insert into many-to-many mapping table (see my previous post)

So here’s a solution. First, create a sequence (available in SQL Server 2012 or later, and in SQL Azure):

create sequence dbo.OrderId as int
start with 1
increment by 1

Then, read from it using the built-in stored procedure:

exec sys.sp_sequence_get_range @sequence_name, @range_size, @range_first_value OUT" 

And finally, execute the stored procedure programmatically. In this particular example, using Entity Framework:

 public async Task<int> GetSequenceStart(string sequenceName, int count)
{
    var sequenceNameParam = new SqlParameter
    {
        ParameterName = "@sequence_name",
        SqlDbType = SqlDbType.NVarChar,
        Direction = ParameterDirection.Input,
        Value = sequenceName
    };
    var countParam = new SqlParameter
    {
        ParameterName = "@range_size",
        SqlDbType = SqlDbType.Int,
        Direction = ParameterDirection.Input,
        Value = count
    };
    var outputParam = new SqlParameter
    {
        ParameterName = "@range_first_value",
        SqlDbType = SqlDbType.Variant,
        Direction = ParameterDirection.Output
    };

    await _dbContext.ExecuteSqlCommandAsync(
        "exec sys.sp_sequence_get_range @sequence_name, @range_size, @range_first_value OUT",
        sequenceNameParam,
        countParam,
        outputParam);

    return (int)outputParam.Value;
}

The drawbacks of this solution is that it doesn’t scale that well:
– if you’re using more than one database, you have to have one of them as a master and it will receive higher load.
– each application instance must be fully aware of such hierarchy what crosses the boundaries of horizontal scaling (aka scaling out).

Happy sequencing!

How to push NuGet package to VSO feed v3

To push a nuget package to Visual Studio Online feed v3 use the following command:

nuget.exe push MyPackage.1.0.0.0.nupkg
    -Source https://mpsit.pkgs.visualstudio.com/DefaultCollection/_packaging/My_Feed/nuget/v3/index.json
    -ConfigFile nuget.config
    -ApiKey My_Feed

The crucial part here is to include -ApiKey My_Feed, otherwise you’ll get 404 Not Found error.

Happy pushing!

How to map linking table for many-to-many relationship using Entity Framework 6

TL;DR: use synonym.

As promised earlier to myself, I blog about interesting challenges and the ways I solved them.

The other day, me and my team faced a limitation in EF6: you can either configure a linking table for many-to-many relationship between two entities or map this table to an entity. But not both simultaneously.

For instance, for the relationship between Orders and Addresses, when one order can be linked to multiple addresses (shipping, billing, legal, etc.) and one address can be used for multiple orders.

modelBuilder.Entity<Order>
            .HasMany(x => x.Addresses)
            .WithMany(x => x.Orders)
            .Map(c => c.ToTable("OrderAddresses")
                       .MapLeftKey("OrderId")
                       .MapRightKey("AddressId"));

And in the same time you’d like to have the intermediate, linking entity to be mapped too. To query it independently or for bulk operations such as insert.

modelBuilder.Entity<OrderAddress>
            .ToTable("OrderAddresses");

This case you’ll get an exception during context configuration validation:

System.InvalidOperationException: The navigation property ‘Addresses’ declared on type ‘Order’ has been configured with conflicting mapping information.

The solution would be to create a synonym (which in my project we keep in a separate schema called syn):

create schema syn
create synonym syn.OrderAddresses for dbo.OrderAddresses

and map the intermediate entity to it:

modelBuilder.Entity<OrderAddress>
            .ToTable("OrderAddresses", "syn");

Note the overload accepting the schema name.

Now you can bulk insert into this table directly:

using (var scope = new TransactionScope())
{
    dbContext.BulkInsert(orders);
    dbContext.BulkInsert(addresses);
    dbContext.BulkInsert(orderAddresses);

    scope.Complete();
}

We’re using EntityFramework.BulkInsert but also you can try EntityFramework.Utilities that supports both bulk insert and delete. However it also has a bug and doesn’t expose an overload accepting SqlBulkCopyOptions.

Happy mapping!

How to convert Google API Service Account certificate to base64

Recently in Google Developer Console by mistake I generated a new SSL certificate for my project’s Service Account so had to convert it again from p12 file to base64 representation and store its thumbprint separately.

Here’s how you can do it:

var cert = new X509Certificate2("project-595a8425b1d7.p12", "notasecret", X509KeyStorageFlags.Exportable);
var thumbprint = cert.Thumbprint;
var base64 = Convert.ToBase64String(cert.Export(X509ContentType.Pfx, "notasecret")); // or Pkcs12 which works as well

And just to make sure output string actually works and certificate contains private key:

var test = new X509Certificate2(Convert.FromBase64String(base64), "notasecret");
Debug.Assert(test.PrivateKey != null);

How to extract private key from pfx and remove passphrase using OpenSSL

When I tried to enable SSL for BitTorrent Sync installed on my new NAS Synology 215j it turned out it requires not pfx but private and public keys separately in base64 encoded form.

Here’s the command to extract certificate itself. It will prompt for existing pfx’s passphrase (password):

openssl pkcs12 -in synology.pfx -clcerts -nokeys -out synology.cer

To extract private key. It will prompt for pfx’s passphrase and for a passphrase to add to the key:

openssl pkcs12 -in synology.pfx -nocerts -out synology.private.key

To remove the later passphrase. Now private key doesn’t contain any:

openssl rsa -in synology.private.key -out synology.key

Troubleshooting site-to-site connection in Azure: error 797, 663.

When I created a site-to-site connection in Azure Networks (classic) and tried to connect to the gateway from my on-premise VPN server running on virtual Windows Server 2012 R2 for the first time I found the following error in Application Event Log:

CoId={guid}: The user SYSTEM dialed a connection named {name} which has failed. The error code returned on failure is 797.

Solution: in RRAS -> Ports -> Properties: WAN Winiport (IKEv2) make sure you have at least 1 port enabled.

Next error I got was:

CoId={guid}: The user SYSTEM dialed a connection named {name} which has failed. The error code returned on failure is 663.

Solution: in the same settings make sure you have Demand-dial routing connection (inbound and outbound) enabled.

Happy routing!