Azure

We use terraform a lot at work and reading some of the terraform configuration or modules and the huge number of variables that sometimes exist, I start to question whether I should just use the underlying resource directly instead of the module abstraction. It comes from the need to paramaterise everything. You get so many variables that it seems like you’ve built another domain specific type that is more complex than the pieces you’re abstracting. ...

Azure functions can be very cheap and very easy to manage. Hosting a single page app (SPA) that does very few requests to the host and is basically just a few files to be delivered to the client seems perfect for using the consumption pricing of azure functions. Basically only pay for a request vs a monthly fee. Azure Resources We’ll need a few resources in Azure, they are: Azure Function app Azure storage account with a blob container that has public access These are very straight forward to create in azure and plenty of articles if you haven’t done this before. ...

I’m a big advocate of the cloud and it’s ability to provide just enough resources ad hoc. You can use whatever you want, and pay for it just when using it. In machine learning there are services such as Google’s ML Engine or Azure’s upcoming Batch AI but during development, data preprocessing etc sometimes you want immediate iterative processes. In these cases, you can’t go past a Jupyter notebook and in this case, running that on a VM. In this post, I’ll outline how I’ve setup such an environment in Azure, focusing on ability to build it up and tear it down via the CLI and using a cheaper VM during development and an easy jumpt to then being able to run it on a GPU machine once things are running smoothly. ...

PowerShell 6.0 has now become generally available and with it the ability to use PowerShell cross platform. With all of its goodness, I was most excited about Azure powershell cross platform capabilities that this brings about. Firstly, the azure-cli is also a great cross platform tool, but sometimes the powershell capabilities seem to outshine this CLI. Installing PowerShell on macOS The installation instructions on the repo are pretty straight forward, but for summary: ...

It was great to see you can now use Ghost as an NPM module. The upgrade process used to be a pain prior to the 1.0 release. Saying this, I ran into a few bumps trying to host this on Azure. Follow the docs on Ghost as an NPM module to get started Set server port at runtime Node runs behind IIS and is reverse proxied so the port we are assigned is not static. In your index.js as setup in 1. add a line to set the port from the environment variable, so your ghost server start lines are as so: ...

I recently completed Stanford Machine Learning on Coursera taught by Andrew Ng. It is a fantastic course for anyone interested, plus it’s free! The course uses MATLAB or Octave for the programming assignments. With the release of Azure Machine Learning, this seemed like a good exercise to reimplement these assignments on Azure ML to learn the tool. Creating an Azure ML Workspace To get started, you will need an Azure subscription. From the Azure Management Portal click ‘+ NEW’ from the lower left and create a workspace, and link to or create an associated storage account for use with the data. ...

Azure Batch AI provides us the PaaS opportunity to use GPU resources in the cloud. The basis is to use virtual machines in a managed cluster (i.e. you don’t have to maintain them) and run jobs as you see fit. For my use case, the opportunity of low-priority VMs to reduce the cost of using GPU machines is also particularly promising. What I’ll run through is running our first job on Azure Batch AI. For setup, we’ll use the Azure CLI as I find it easier and quicker than using the UI portal. Saying that, everything can be achieved by point and click at portal.azure.com Assuming you already have the CLI installed and you are already logged in with it. ...

Azure provides low priority VMs that we can use to save some money when utilising GPUs for machine learning. All the major cloud providers offer spare compute for a discount: Google cloud has Preemptible VMs, AWS has spot instances and Azure has low priority VMs. On Azure these low priority VMs can only be provisioned in Azure Batch or virtual machine scale sets and we can use the latter for hosting a jupyter environment. The low priority pricing is 23c/hour vs the normal $1.147/hour…a nice saving ...

I love getting to a point with Infrastructure as Code (IaC) where not only are the resources reproducable, but also encoding good security and utilisation of cloud resources into the contents. Firstly, support in Azure Storage for Active Directory access control went GA and utilising this over an access key is one of those security considerations that seems could be automated. Secondly, managed identities are a fantastic way to get the power of Azure Active Directory without the process of keeping secrets and other management secure. ...