Engineering and technology

Introduction to cloud-native applications

Note how the application is decomposed across a set of small isolated microservices. Each service is self-contained and encapsulates its own code, data, and dependencies. Each is deployed in a software container and managed by a container orchestrator. Instead of a large relational database, each service owns it own datastore, the type of which vary based upon the data needs. Note how some services depend on a relational database, but other on NoSQL databases. One service stores its state in a distributed cache. Note how all traffic routes through an API Gateway service that is responsible for routing traffic to the core back-end services and enforcing many cross-cutting concerns. Most importantly, the application takes full advantage of the scalability and resiliency features found in modern cloud platforms.

https://docs.microsoft.com/en-us/dotnet/architecture/cloud-native/introduction

Azure Storage Replication Explained

In the previous post about Storage Accounts, we talked about various types of accounts and their associated data services. Since that post, the Premium File storage option that was in preview has now gone into General Availability. (https://azure.microsoft.com/en-us/blog/announcing-the-general-availability-of-azure-premium-files/)

Alongside the data access tiers (Hot, Cool, and Archive) and data services (Blob, File, Queue, and Table), Azure also has options around the data redundancy. In this post, we are going to look at the four types of data replication Azure provides.
LRS (Locally redundant storage)

The most ubiquitously available option is Locally Redundant Storage (LRS); this is the default and only replication type available for all storage account types.

LRS ensure your data is replicated three times within a single data centre. These datastores are updated using synchronous writes to guarantee all three copies are kept up to date.

LRS does have downsides, predominately due to a single data centre in a single azure region containing the data replication. This issue exposes the data to a single point of failure if the Data Centre is entirely offline. Microsoft does commit to a 99.9% SLA for read and writes operations for data stored in LRS datastores. The SLA is not to be confused with the 11 9’s (99.999999999%) guarantee they offer for data durability which is just a commitment to ensure a level of data integrity against data loss or corruption.
ZRS (Zone redundant storage)

Introduced for General Purpose v2 storage accounts Zone Redundant Storage (ZRS) brings enhancements to the replication seen in LRS.

As with LRS, ZRS has synchronous writes between 3 copies of your data to ensure data integrity. Although still within the same Azure region, additional resilience is introduced due to the use of availability zones within the region. Two or three availability zones contain copies of the data. The increased resilience removes the issues of a single data centre outage causing data access issues. Although the data is spread across multiple availability zones, these zones are all within a single region, so data available is still susceptible to region-wide outages.

ZRS has the same SLA levels for read and write operations as LRS, but they increase the integrity of the durability of the data objects by an additional 9 to 99.9999999999% (12 9’s).
GRS (Geo-redundant storage)

Geo-redundant storage (GRS) brings additional redundancy to the data storage over both LRS or ZRS. Along with the three copies of your data stored within a single region, a further three copies are stored in the twinned Azure region. So using GRS means you get all the features of the LRS storage within your primary zone, but you also get a second LRS data storage in a neighbouring Azure region. This data is updated asynchronously, so there is a small lag between the 2 data sets, but for most cases this is acceptable.

Although using GRS means you are using two different datacentres in conjunction, there is a drawback of GRS, which is the secondary data storage is not accessible to read unless the storage account fails over. Due to all read and write operations still being managed by via a single data centre, Microsoft offers the same read and write SLA’s as ZRS and LRS datastores.
RA-GRS (Read-Access Geo-redundant)

The final replication type available is Read Access Geo-redundant storage, it has all the benefits and redundancy of the standard GRS replication but also allows for the secondary copy stored in the paired sister Azure region to be readable. This means you have multiple endpoints that are readable that your applications can use if they are configured to handle this configuration.

The additional read endpoint also means that for RA-GRS the read operation SLA is increased to 99.99% availability for Hot datastores. Write operations are left at the 99.9% SLA due to the single region still being in control of the write and update operations.

Both types of GRS replication do have a slight delay in the replication due to its asynchronous behaviour. This is where checks of the LastSyncTime come in useful to ensure you are reading the most up to date copy of the data. This asynchronous replication needs to be checked against your Recovery Point Objectives if you are planning to use GRS / RA-GRS as part of your DR planning.

Currently, Microsoft controls the failover of storage accounts from one Azure region to another region, so the scope for using the failover on your terms is limited. Microsoft has recently, however, brought into preview an option for a customer-initiated failover themselves. This is currently only available in the US West 2 / US West Central region pair and while in preview not suitable for production environments.

What is the difference between an Azure tenant and Azure subscription?

Basic understanding:

  • a tenant is associated with a single identity (person, company, or organization) and can own one or several subscriptions
  • a subscription is linked to a payment method
  • in every subscription, you can add virtual resources (VM, storage, network, …)

Additionally:

  • Every tenant is linked to a single Azure AD instance, which is shared with all tenant’s subscriptions

  • Resources from one subscription are isolated from resources in other subscriptions

  • An owner of a tenant can decide to have multiple subscriptions:

    • when Subscriptions limits are reached
    • to use different payment methods
    • to isolate resources between different entities (departments)

Example 1:

Contoso decides to have a tenant with 2 subscriptions:

  • one subscription for the Prod department with Credit Card A
  • one subscription for the Dev department with Credit Card B
    (but could also be the same Credit Card as the one of another subscription)

In this example, the two departments share the same Azure AD database. However, resources are isolated between departments, and budgets can be separated too.

Example 2:

A holding company decides to have 2 tenants:

  • one tenant for subsidiary Contoso with one subscription for Dev and Prod
  • one tenant for subsidiary Fabrikam with one subscription for Dev and another subscription for Prod

In this example, both companies have a different Azure AD database.

Example 3:

You have a tenant for your personal training.
In this tenant, you can have:

  • one free Azure subscription (linked to a credit card but not charged, and can be converted to a Pay-As-You-Go subscription after the free trial)
  • one or several Pay-As-You-Go subscriptions (linked to different credit cards)
  • one or several Azure Pass Sponsorship subscriptions, not linked to any credit card because these subscriptions are obtained during Microsoft trainings
  • one Visual Studio subscription (linked to a credit card) and with different quotas (of free resources) than the free subscription

Despite all those subscriptions have isolated resources (per subscription), and some are free while you have to pay for others, all subscriptions share the same Azure AD database.

https://stackoverflow.com/questions/47307368/what-is-the-difference-between-an-azure-tenant-and-azure-subscription

Blob – Binary large object

A Binary Large OBject (BLOB) is a collection of binary data stored as a single entity in a database management system. Blobs are typically images, audio or other multimedia objects, though sometimes binary executable code is stored as a blob. Database support for blobs is not universal.

Blobs were originally just big amorphous chunks of data invented by Jim Starkey at DEC, who describes them as “the thing that ate Cincinnati, Cleveland, or whatever” from “the 1958 Steve McQueen movie”, referring to The Blob. Later, Terry McKiever, a marketing person for Apollo, felt that it needed to be an acronym and invented the backronym Basic Large Object. Then Informix invented an alternative backronym, Binary Large Object.

The data type and definition was introduced to describe data not originally defined in traditional computer database systems, particularly because it was too large to store practically at the time the field of database systems was first being defined in the 1970s and 1980s. The data type became practical when disk space became cheap. This definition gained popularity with IBM‘s DB2.

The term is used in NoSQL databases, especially in Key-value store databases such as Redis.

The name “blob” is further borrowed by the deep learning software Caffe to represent multi-dimensional arrays.

In the world of free and open-source software, the term is also borrowed to refer to proprietary device drivers, which are distributed without their source code, exclusively through binary code; in such use, the term binary blob is common, even though the first letter in the blob abbreviation already stands for binary

https://en.m.wikipedia.org/wiki/Binary_large_object

Disk Stripping – Optimize storage performance

Disks can be striped using a striping technology (such as Storage Spaces on Windows or mdadm on Linux) to increase the throughput and IOPS by spreading disk activity across multiple disks. Using disk striping allows you to really push the limits of performance for disks, and is often seen in high-performance database systems and other systems with intensive storage requirements.

https://docs.microsoft.com/en-us/learn/modules/design-for-performance-and-scalability-in-azure/4-optimize-storage-performance

Optimize storage performance – Premium Storage

Premium storage can attach only to specific virtual machine (VM) sizes. Premium storage capable sizes are designated with an “s” in the name, for example D2s_v3 or Standard_F2s_v2. Any virtual machine type (with or without an “s” in the name) can attach standard storage HDD or SSD drives.

https://docs.microsoft.com/en-us/learn/modules/design-for-performance-and-scalability-in-azure/4-optimize-storage-performance

Virtualbox – Teleporting Virtual Machines

Oracle VM VirtualBox supports teleporting. Teleporting is moving a virtual machine over a network from one Oracle VM VirtualBox host to another, while the virtual machine is running. This works regardless of the host operating system that is running on the hosts. You can teleport virtual machines between Oracle Solaris and Mac OS X hosts, for example.

Teleporting requires that a machine be currently running on one host, which is called the source. The host to which the virtual machine will be teleported is called the target. The machine on the target is then configured to wait for the source to contact the target. The machine’s running state will then be transferred from the source to the target with minimal downtime.

Teleporting happens over any TCP/IP network. The source and the target only need to agree on a TCP/IP port which is specified in the teleporting settings.

At this time, there are a few prerequisites for this to work, as follows:

  • On the target host, you must configure a virtual machine in Oracle VM VirtualBox with exactly the same hardware settings as the machine on the source that you want to teleport. This does not apply to settings which are merely descriptive, such as the VM name, but obviously for teleporting to work, the target machine must have the same amount of memory and other hardware settings. Otherwise teleporting will fail with an error message.
  • The two virtual machines on the source and the target must share the same storage, hard disks as well as floppy disks and CD/DVD images. This means that they either use the same iSCSI targets or that the storage resides somewhere on the network and both hosts have access to it using NFS or SMB/CIFS.

    This also means that neither the source nor the target machine can have any snapshots.

To configure teleporting, perform the following steps:

  1. On the target host, configure the virtual machine to wait for a teleport request to arrive when it is started, instead of actually attempting to start the machine. This is done with the following VBoxManage command:
    VBoxManage modifyvm targetvmname --teleporter on --teleporterport port

    targetvmname is the name of the virtual machine on the target host and port is a TCP/IP port number to be used on both the source and the target hosts. For example, use 6000. See Section 8.8, “VBoxManage modifyvm”.

  2. Start the VM on the target host. Instead of running, the VM shows a progress dialog, indicating that it is waiting for a teleport request to arrive.
  3. Start the VM on the source host as usual. When it is running and you want it to be teleported, issue the following command on the source host:
    VBoxManage controlvm sourcevmname teleport --host targethost --port port

    where sourcevmname is the name of the virtual machine on the source host, which is the machine that is currently running. targethost is the host or IP name of the target host on which the machine is waiting for the teleport request, and port must be the same number as specified in the command on the target host. See Section 8.13, “VBoxManage controlvm”.

For testing, you can also teleport machines on the same host. In that case, use localhost as the hostname on both the source and the target host.

http://www.virtualbox.org/manual/ch07.html#teleporting