Carnation Anapa Winery, vol 3, day 4: yeast

Due to the pandemic and workaholism, everything takes longer this year.

I’m adding 5g of RC212 by Cellar Science (batch #52495, whatever it means) to the 5-gallon bucket of Petit Sirah. But first, to avoid shock, I’m diluting the yeast in a small amount of boiled water cooled down to 106°F.

Posted in Winemaking | Tagged | Leave a comment

Carnation Anapa Winery, vol 3, day 3: Potassium Metabisulfite

Last time when I added Potassium Metabisulfite the outcome was much better when I did not. So this time I’m adding it to both buckets of must, ~1.5 x ¼ tsp per 5 gallons.

Posted in Winemaking | Tagged | Leave a comment

Carnation Anapa Winery, vol 3, day 2: weighing

Some precalculations:

  • My weight: 74.15 kg
  • Empty bucket: 1.15 kg
  • Total: 75.45 kg

Bucket #1 (CS):

  • Total: 86.80 kg
  • Grapes: 12.65 kg

Bucket #2 (CS):

  • Total: 85.80 kg
  • Grapes: 11.65 kg

Bucket #3 (PS):

  • Total: 89.15 kg
  • Grapes: 15.00 kg

Bucket #4 (PS):

  • Total: 90.05 kg
  • Grapes: 15.9 kg

What in sum runs as:

  • Cabernet Sauvignon: 24.3 kg
  • Petite Sirah: 30.9 kg
  • Total: 55.2 kg (121.695 lbs)
Posted in Winemaking | Tagged | Leave a comment

Carnation Anapa Winery, vol 3, day 1: The journey continues

It’s that time of year when I drive to my friends at Carthage Vineyard in Zillah, WA and pick what’s left after the harvest season.

This year it was 2 buckets of Cabernet Sauvignon and 2 buckets of Petite Sirah.

Posted in Winemaking | Tagged | Leave a comment

How to configure Service Fabric to use AAD for client authentication

This blob post is intended to compliment the official doc which I personally don’t find helpful and comprehensive enough.

The configuration that works for me consists of 3 parts:

  1. Cluster ARM template change
  2. AAD app for the cluster identity (let’s call it client)
  3. AAD app for the users to access SFE (let’s call it cluster)

First you make the changes in your ARM template for the cluster and deploy:

"variables": {
  "clientAadAppId": "{client app id}",
  "clusterAadAppId": "{cluster app id}"
},
"resources": [
  {
    "type": "Microsoft.ServiceFabric/clusters",
    "apiVersion": "[variables('sfApiVersion')]",
    "name": "[parameters('clusterName')]",
    "location": "[parameters('location')]",
    "properties": {
      "addonFeatures": [],
      "azureActiveDirectory": {
        "tenantId": "[subscription().tenantId]",
        "clientApplication": "[variables('clientAadAppId')]",
        "clusterApplication": "[variables('clusterAadAppId')]"
      },
      "certificateCommonNames": {},
      "clientCertificateCommonNames": [],
      "clientCertificateThumbprints": [],
      "diagnosticsStorageAccountConfig": {},
      "fabricSettings": [],
      "managementEndpoint": "",
      "reliabilityLevel": "[variables('reliabilityLevel')]",
      "upgradeMode": "Automatic",
      "vmImage": "Windows"
    }
  }
]

Then you create 2 third-party AAD applications and edit their manifests.

For the client app where you specify the Microsoft Graph and cluster app ids:

"requiredResourceAccess": [
  {
    "resourceAppId": "00000003-0000-0000-c000-000000000000",
    "resourceAccess": [
      {
        "id": "{random guid}",
        "type": "Scope"
      }
    ]
  },
  {
    "resourceAppId": "{cluster app id}",
    "resourceAccess": [
      {
        "id": "{your guid}",
        "type": "Scope"
      }
    ]
  }
],
"oauth2Permissions": [
  {
    "adminConsentDescription": "Allow the application to access SF Cluster Management application on behalf of the signed-in user.",
    "adminConsentDisplayName": "Access SF Cluster",
    "id": "{your guid}",
    "isEnabled": true,
    "lang": null,
    "origin": "Application",
    "type": "User",
    "userConsentDescription": "Allow the application to access SF Cluster Management application on your behalf.",
    "userConsentDisplayName": "Access SF Cluster",
    "value": "user_impersonation"
  }
]

And for the cluster app where you specify what roles have what permissions:

"appRoles": [
  {
    "allowedMemberTypes": [
      "User"
    ],
    "description": "ReadOnly roles have limited access",
    "displayName": "ReadOnly",
    "id": "{random guid}",
    "isEnabled": true,
    "lang": null,
    "origin": "Application",
    "value": "User"
  },
  {
    "allowedMemberTypes": [
      "User"
    ],
    "description": "Admins roles can perform all tasks",
    "displayName": "Admin",
    "id": "{random guid}",
    "isEnabled": true,
    "lang": null,
    "origin": "Application",
    "value": "Admin"
  }
]

And finally add your cluster’s SFE endpoint to the the Authentication section

https://{clusterName}.{clusterLocation}.cloudapp.azure.com:19080/Explorer/index.html

And finally go to the cluster app Overview and click Managed application in local directory, select Users and Group and assign permissions to your AAD groups you want to be Users or Admins.

That’s it, folks!

Posted in Infrastructure | Tagged | Leave a comment

How to hook up child DNS zone into parent by updating its NS records using ARM template

Imagine a scenario: you have one global DNS zone in Prod subscription and several child DNS zones for each environment in their own subscriptions, e.g.:

  • infra.example.com
    • Subscription: Prod
  • dev.infra.examle.com
    • Subscription: Dev
  • test.infra.example.com
    • Subscription: Test
  • prod.infra.example.com
    • Subscription: Prod

Each zone is created using its own ARM template. But in order a child zone to start working you need to hook it up into the parent zone by updating its NS record, e.g.:

  • dev.infra.example.com
    • NS
      • ns1-01.azure-dns.com.
      • ns1-01.azure-dns.net
      • ns1-01.azure-dns.org.
      • ns1-09.azure-dns.info.
  • infra.example.com
    • dev
      • NS
        • the records must be inserted here

Here’s how to achieve that using ARM template:

{
  "name": "[parameters('envDnsZoneName')]",
  "type": "Microsoft.Network/dnsZones",
  "apiVersion": "[variables('publicDnsApiVersion')]",
  "location": "global"
},
{
  "name": "[concat('DNS-Global-', parameters('environment'))]",
  "type": "Microsoft.Resources/deployments",
  "apiVersion": "[variables('apiVersion')]",
  "subscriptionId": "[parameters('globalSubscriptionId')]",
  "resourceGroup": "[parameters('globalResourceGroupName')]",
  "properties": {
    "mode": "Incremental",
    "template": {
      "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
      "contentVersion": "1.0.0.0",
      "resources": [
        {
          "type": "Microsoft.Network/dnsZones/NS",
          "name": "[concat(parameters('globalDnsZoneName'), '/', parameters('environment'))]",
          "apiVersion": "[variables('publicDnsApiVersion')]",
          "properties": {
            "TTL": 3600,
            "NSRecords": "[reference(resourceId('Microsoft.Network/dnszones/NS', parameters('envDnsZoneName'), '@'), variables('publicDnsApiVersion')).NSRecords]"
          }
        }
      ]
    }
  },
  "dependsOn": [
    "[concat('Microsoft.Network/dnsZones/', parameters('envDnsZoneName'))]"
  ]
}

Here’s what it does:

  1. Creates a child zone in current subscription and resource group
  2. Updates the parent zone in its own subscription and resource group, creates NS record with the value of NS records of the child zone

Happy deployment!

Posted in Infrastructure | Tagged , | Leave a comment

How to enable automatic clean up of provisioned application types on a Service Fabric cluster

As time goes by and you deploy applications, a new build every time what means a new application type is getting provisioned. Application packages are piling up and after some time old versions become just a clutter that eats up disk space without providing any value. So you may want to periodically clean them up.

Before you had to periodically run a PowerShell script manually, like this one:

param
(
  [Parameter(Mandatory=$true)]
  [string]$ApplicationTypeName,

  [Parameter(Mandatory=$true)]
  [int]$NumberOfTypesToKeep
)

Set-ExecutionPolicy -ExecutionPolicy Unrestricted -Scope CurrentUser -Force

$moduleName = "ServiceFabric"
$module = Get-Module $moduleName
if (!$module)
{
  Write-Output "Module $moduleName was not found, importing"
  Import-Module "C:\Program Files\Microsoft Service Fabric\bin\ServiceFabric\ServiceFabric.psd1"
}

Connect-ServiceFabricCluster -Verbose

if ($ApplicationTypeName -eq "*")
{
  $applications = Get-ServiceFabricApplicationType -Verbose
  $applicationTypeNames = $applications.ApplicationTypeName | Select-Object -Unique
}
else
{
  $applicationTypeNames=$ApplicationTypeName
}

foreach ($appTypeName in $applicationTypeNames)
{
  $currentAppType = Get-ServiceFabricApplication -ApplicationTypeName $appTypeName -Verbose

  # if type not found
  $registeredAppTypes = Get-ServiceFabricApplicationType -ApplicationTypeName $appTypeName -Verbose
  if (!$registeredAppTypes)
  {
    Write-Error "Application Type '$appTypeName' was not found, skipped cleanup"
    continue
  }

  $registeredVersions = $registeredAppTypes.ApplicationTypeVersion

  # if to keep > total
  if ($NumberOfTypesToKeep -ge $registeredVersions.Count)
  {
    Write-Error "Parameter NumberOfTypesToKeep=$NumberOfTypesToKeep is greater than or equals the number of registered types=$($registeredVersions.Count)"
    continue
  }

  $versionsToDelete = $registeredVersions | `
                      Where-Object { $currentAppType.ApplicationTypeVersion -notcontains $_ } | `
                      Select-Object -First ($registeredVersions.Count - $NumberOfTypesToKeep)

  Write-Output "Application type '$appTypeName' does exist, started deletion"
  foreach ($versionToDelete in $versionsToDelete)
  {
    Unregister-ServiceFabricApplicationType -ApplicationTypeName $appTypeName `
                                            -ApplicationTypeVersion $versionToDelete `
                                            -Force `
                                            -Verbose
  }

  Write-Output "Successfully deleted application type '$appTypeName' versions: $versionsToDelete"
}

But since version 6.5 you don’t need to do anything manual anymore. Here’s a snippet from cluster ARM template:

{
  "name": "Management",
  "parameters": [
    {
      "name": "CleanupUnusedApplicationTypes",
      "value": true
    },
    {
      "name": "PeriodicCleanupUnusedApplicationTypes",
      "value": true
    },
    {
      "name": "TriggerAppTypeCleanupOnProvisionSuccess",
      "value": true
    },
    {
      "name": "MaxUnusedAppTypeVersionsToKeep",
      "value": "10"
    }
  ]
}

That’s it folks, happy deployment!

Posted in Infrastructure | Tagged , , | Leave a comment

How to use EndpointCertificate with proper ACL for Network Service in Service Fabric

Presentiments: your Service Fabric cluster must be of version 7.1. Since the automatic rollout was postponed, you would need to temporarily switch to the manual mode, select the latest version, and then switch back once the upgrade is complete. To me it took just a few minutes.

undefinedundefined

Before I had the following configuration for a SSL certificate in my application manifest. The endpoint certificate was configured as regular certificate because Service Fabric was not supporting loading it by SNI, and now it does. I also had manually to ACL it to Network Service.

  <Principals>
    <Users>
      <User Name="NetworkServiceUser" AccountType="NetworkService" />
    </Users>
  </Principals>
  <Policies>
    <SecurityAccessPolicies>
      <SecurityAccessPolicy ResourceRef="SSLCert" PrincipalRef="NetworkServiceUser" GrantRights="Full" ResourceType="Certificate" />
    </SecurityAccessPolicies>
  </Policies>
  <Certificates>
    <SecretsCertificate X509FindType="FindBySubjectName" X509FindValue="[SSL_Certificate_SubjectName]" Name="SSLCert" />
  </Certificates>

Now it looks like this:

<ServiceManifestImport>
  <Policies>
    <EndpointBindingPolicy EndpointRef="ServiceTypeEndpoint_Secure" CertificateRef="SSLCert" />
  </Policies>
</ServiceManifestImport>
<Certificates>
  <EndpointCertificate X509FindType="FindBySubjectName" X509FindValue="[SSL_Certificate_SubjectName]" Name="SSLCert" />
</Certificates>

Once the application is deployed, Service Fabric will start looking for a new certificate by SNI every 1 minutes and ACL it to Network Service (grant it access to the private key) automatically.

Posted in Programming | Tagged | Leave a comment

The benefits of using Arrays over Lists in C#

I’m an ardent proponent of using Arrays (T[]) over Lists (Lisf<T>) in C#. In my case, it’s mostly web services running in Azure and libraries, consumed by such services. In this blog post I’ll be collecting the pros and cons of using one over another.

First and foremost, the methodological argument. Types matter, whenever it’s a parameter type or a return type. There are many types but this one is mine. Type indicates the intend.

What’s the intend of a list? A list is a collection that allows an item to be added:

List<T>.Add(T)

What means a list is not a final collection. Its “true name” is ArrayList. That it, the IList interface implemented by using an Array. Historically, C# design has copied so many things from Java but unfortunately this class name one was not one of them. Its core functionality pivots around automatic resizing (you can arbitrarily add and remove items) and capacity.

While an array is the opposite. It’s final. Once created, its size cannot be changed. Of course, you can resize it but that will be a new array with the content copied over. Like strings, arrays are immutable in that regards.

Side note: there are already readonly lists and collections, as well immutable arrays. There is a number of problems with them:

  • Fom a practical standpoint of the cost of writing and maintaining the code. In general, programmers are too lazy and the name is too long. How do you like Task<IReadOnlyCollection<OrderDetails>> comparing to just Task<OrderDetails[]>? Because C# is notoriously “chatty” aka high ceremony language.
  • From a technical standpoint of writing a better, more efficient code at the baseline, means from the beginning. Both types T[] and List<T> are optimized for enumeration and return a struct from GetEnumerator(), if called on the concrete class. As soon as it’s called on the interface (IEnumerable<T> or any of its descendants) the struct is boxed and all the benefits disappear.

As I mentioned above, I’m using C# to write web services in the cloud. What means it’s strings from a database sent over a network. The result of a query? Final. The results in a HTTP request? Final as well.

Then why would your Database-over-Network class return a type whose intend is to add stuff into it?

interface IOrderService
{
    Task<List<OrderDetails>> GetOrders();
}

Obviously, it should be an array instead.

Here we could stop. In my mind, this argument alone is enough to always use arras until you clearly need otherwise. But let’s discuss other, technical arguments supporting my position.

  1. What happens when you’re instantiating a generic List<T>? “The whole transitive closure of types, starting with that root type, will be compiled”. See Joe Duffy’s blog post on this topic. Generics are not bad but they’re relatively expensive.
  1. Accessing an item inside an array involves a bounds checking. Which are in many cases eliminated by JIT optimization down the road anyway. See this question on Stack Overflow and Matt Warren’s blog post.The code for array indexer is getting translated into IL directly:
IL_0014: ldelem.i4

While list indexer is a whole virtual method call:

IL_001b:  callvirt   instance !0 class [mscorlib]System.Collections.Generic.List`1<int32>::get_Item(int32)
  1. Task<T> is not covariant. What means

To be continued….

Posted in Programming | Tagged | Leave a comment

How to set access policy on Key Vault in another subscription aka How to deploy to resource group in another subscription using ARM template?

The model for my Service Fabric infrastructure consists of two major parts:

  • Data Center (think Azure region)
  • Scale Unit (think Service Fabric cluster and its child resources)

But today due to the limitation around AAD first party application we decided to add one more level on top of it:

  • Global secrets (think 1PA token encryption/decryption certificate)

What means the Key Vault extension now needs to download yet another certificate from a global Key Vault. It will use User-Assigned Identity to authorize on this Key Vault. What means the Key Vault needs to have the appropriate access policies. But here’s the challenge:

  • Subscription 1
    • Resource Group 1 (Global Secrets)
      • Global Key Vault
  • Subscription 2
    • Resource Group 2 (DC)
      • Regional Key Vault
    • Resource Group 3 (SU)
      • Service Fabric cluster
      • VMSS
        • Key Vault extension runs here

What means the ARM template for UAID needs somehow set the access policy for in on a Key Vault not just in another resource group but in another subscription altogether. Here’s how to do that:

{
  "$schema": "http://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "uaidName": {
      "type": "string"
    },
    "globalSecretsSubcriptionId": {
      "type": "string"
    },
    "globalSecretsResourceGroupName": {
      "type": "string"
    },
    "globalSecretsKvName": {
      "type": "string"
    }
  },
  "variables": {
    "apiVersion": "2019-09-01",
    "idApiVersion": "2018-11-30",
    "kvApiVersion": "2019-09-01",
    "tenantId": "[subscription().tenantId]",
    "uaidRef": "[concat('Microsoft.ManagedIdentity/userAssignedIdentities/', parameters('uaidName'))]"
  },
  "resources": [
    {
      "name": "[concat('KeyVault-Global-', parameters('uaidName'))]",
      "type": "Microsoft.Resources/deployments",
      "apiVersion": "[variables('apiVersion')]",
      "subscriptionId": "[parameters('globalSecretsSubscriptionId')]",
      "resourceGroup": "[parameters('globalSecretsResourceGroupName')]",
      "properties": {
        "mode": "Incremental",
        "template": {
          "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
          "contentVersion": "1.0.0.0",
          "resources": [
            {
              "type": "Microsoft.KeyVault/vaults/accessPolicies",
              "name": "[concat(parameters('globalSecretsKvName'), '/add')]",
              "apiVersion": "[variables('kvApiVersion')]",
              "properties": {
                "accessPolicies": [
                  {
                    "tenantId": "[variables('tenantId')]",
                    "objectId": "[reference(variables('uaidRef'), variables('idApiVersion')).principalId]",
                    "permissions": {
                      "keys": [
                      ],
                      "secrets": [
                        "Get"
                      ],
                      "certificates": [
                        "Get"
                      ]
                    }
                  }
                ]
              }
            }
          ]
        }
      }
    }
  ]
}

Posted in Infrastructure | Tagged , | Leave a comment