GCP has recently added support for PHP 7.4 in Cloud Functions. This post will give you a quick introduction to how it works, and a few pasisng thoughts.

If you haven't setup a GCP project before you'll need to go through the sign up process and configure a billing account.

Additionally you'll need to install the gcloud CLI: 

https://cloud.google.com/sdk/docs/install

Once the gcloud cli is installed you'll be able to authenticate with the following command:

Let's start by creating a file named index.php with the following:

Hit save. Then open up a terminal in the same directory. To deploy this function we'll run the following:

Feel free to replace http-php-function with a suitable name for your test function. It needs to be unique within your GCP project.

As this is a HTTP invoked function the first time you deploy a function it will ask you if you want to allow unauthenticated invocations of the functions. Say yes here for this example.

The command above will output various details, including the trigger url. This will look a little like this:

And there it is. Your first PHP function has been deployed on GCP!

If you want to view the logs after invoking the function you can do so with this command:

Note: It might take a minute for logs to appear, as there is a slight delay.

By default, if your application code throws an exception then a 500 status code will be returned, otherwise it'll be treated as a 200.

To send different status codes, and headers we'll need our function to return an instance compatible with Psr\Http\Message\ResponseInterface. We can use Guzzle PSR7 response object for this.

Then our updated function will look like this:

When your function is deployed if it contains a composer.json file then the dependencies will be installed within Cloud Build. If you use a private package registry, then credentials will need to be provided to that context using build environment variables. 

https://cloud.google.com/functions/docs/env-var#using_build_environment_variables

For this you'll want to use the COMPOSER_AUTH environment variable. 

https://getcomposer.org/doc/articles/authentication-for-private-packages.md#authentication-using-the-composer-auth-environment-variable

If the package is listed on packagist then you don't need to do anything else.

Next up I'll touch on local development using GCP's Functions Framework, other ways to invoke functions, and handling application logging.

The best way right now to get going locally is to run GCP's Functions Framework. Let's get started by installing the composer dependency into our existing project:

To run your function locally you can just run this:

Note that the FUNCTION_TARGET environment variable needs to be set, and this needs to match the function you wish to invoke.

You can find out more about the Functions Framework from GitHub: 

https://github.com/GoogleCloudPlatform/functions-framework-php

When you're running serverless stacks you do not have access to the runtime, so it's best to instrument your code with logging to give you plenty of visibility.

GCP has a logging tool called Cloud Logging. In PHP we can write to stderr and the logs will appear in Cloud Logging. You can also use the Cloud Logging SDK.

There are a few drawbacks to using the SDK, namely you need to instrument the invocation id of the function yourself and if you want support with Cloud Trace then you'll also need to include the trace id, as well as specifying the function name and region. Writing to stderr will include all of these attributes automatically. Without them, you'll need to write custom queries to retrieve them from Cloud Logging and they won't appear when viewing logs for your function (it'll need the resource type, function name, and region at the very least).

First up you'll need to install the SDK with composer like so:

Then here's a basic example with the bare minimum to get logs appearing in the right place:

You can grab the function name using the K_SERVICE environment variable, but the region you'd need to configure yourself as GCP Cloud Functions do not expose an environment variable for this. You could add a custom environment variable to grab that value (instead of hard coding, as you might want to deploy to multiple regions).

More info on the logging SDK can be found here: 

https://github.com/googleapis/google-cloud-php-logging

If you want to see a complete example which shows you how to include the function invocation ID, and Cloud Trace ID then here's a gist for you: 

https://gist.github.com/ashsmith/69541dc0e406d767e3fa3de978d66677

At the time of writing Cloud Functions PHP 7.4 supports HTTP and CloudEvents. What are CloudEvents? In esence they allow us to respond to events trigger within GCP. For example Pub/Sub, Cloud Storage or Firestore. CloudEvents use the 

 standard to describe events in a consistent manner.

Let's create a function that responds to messages from Pub/Sub:

Then add to the following function to your existing index.php file:

Now we can deploy the function with the following:

Once thats deployed we can publish a message:

And we can check the logs of our function:

Having first-class support for PHP is fantastic. So far you have only been able to run PHP in serverless functions on AWS and Azure, so support coming to GCP is a good thing!

It seems odd to me that they have added PHP 7.4 support and not PHP 8.0 considering thats the latest release. But this has happened with other runtimes in the past. My guess is we'll have support before 7.4 is deprecated..

The logging leaves a lot to be desired right now too. Documentation suggests that structured logs are possible, but I've not been able to get this to work. I think this is due PHP adding additional text to the output:

The json string should have been parsed by Cloud Logging, but it still treated it as a plain string. Without being able to at least control the severity in this way makes logging and log-based metrics pretty much useless in a production application.

The alternative is to use to logging SDK I mentioned earlier, but this has drawbacks as I mentioned, and would require significant work to get it where you need it to be useful.

I recongise that PHP 7.4 support is in preview, so hopefully these problems will be resolved in the future.

If you want to use PHP and serverless infrastructure I recommend 

, it's for AWS Lambda. It uses the serverless framework and a custom Lambda Layer to provide the PHP runtime.

Cloud Functions and PHP

In this post I'll show you how you can configure SSH keys for your custom Bitbucket Pipe. This requires a tiny bit of setup, as there is no ~/.ssh directory on the docker container. This makes using SSH a little more tricky, especially when you're using tools like git or other deployment tools that hide the actual SSH configuration.

Quick rant: who the heck named custom pipeline tasks "pipes". It's pretty much impossible to Google! I just get pipeline results.

In your shell script you'll just need to add the following:

It's important to copy the id_rsa_tmp AND known_hosts. Without the known hosts file you'll be unable to connect to remote origins. The id_rsa_tmp represents the custom private SSH key that is added into bitbucket pipelines when you follow their guide here: 

https://support.atlassian.com/bitbucket-cloud/docs/variables-and-secrets/#Use-SSH-keys-in-Bitbucket-Pipelines

Make sure ssh is installed on your docker image of course too!

Setting up SSH on custom Bitbucket Pipes

. The reason for doing is so that you can declaritively define certain aspects of BigCommerce configuration in code. At the moment the provider only supports managing webhooks.

Why define webhooks as terraform resources?

This allows us to declaratively define what events are are being consumed, which makes it easy for us to reason our applications infrastructure and it's dependencies. Additionally this terraform code can be applied to multiple environments so that we can ensure they are setup exactly the same.

We can take this a step further when combined with other terraform resources. For example, we can define the destination of our webhook to map directly to a HTTP-triggered GCP Cloud Function that will consume the webhook events:

Just like that we'll have all order events sent to a newly provisioned Cloud Function. This is repeatable, and as mentioned before can be applied to multiple environment to ensure consistency in our configuration.

https://github.com/ashsmith/terraform-provider-bigcommerce/

https://github.com/ashsmith/bigcommerce-api-go/

And the Terraform provider is available in the registry here:

https://registry.terraform.io/providers/ashsmith/bigcommerce/latest

Terraform provider for BigCommerce

Today I created a small CLI utility called ssl-gen. When developing sites or APIs locally, having a trusted SSL certificate locally is always difficult. I can never remember all the parameters, and usually write some horrid bash script. To solve this problem I've created ssl-gen. It will generate your certificate and add it to Keychain so that it is registered as a trusted certificate (getting around those annoying invalid cert errors in your browser!).

Then, to use it you provide your desired certificate name followed by the domains you wish to be on the certificate. The first domain will be the used as the common name.

This will generate ashs-awesome-cert.crt and ashs-awesome-cert.key. These can be used on your webserver such as nginx or apache to serve your local development site on HTTPS.

ssl-gen new also accepts -o / --output which allows you to specify where the certificate will be saved:

Now your certificate will be saved to: ~/.certs/ashs-cert.crt.

Introducing ssl-gen a CLI utility for generating self-signed certificates for macOS

Typescript can be really powerful way of enhancing your react components to ensure the correct props are provided with expected values. Sometimes however you may want some props to required based on what other props have been provided!

Now, our layout prop here might only be able to accept certain values. For example: responsive or fill. So let's convert this example into Typescript:

So now we have correct typed the correct values, what if we wanted to enforce a rule that when layout="fill" is provided you are NOT allowed to define the width or height props. But when layout="responsive" is provided you MUST set them.

Well, to do that we can change our ImageProps type to support exactly this.

Using the Omit utility type we are retrieving all the props that an img element can have, then omitting the src, width and height props. This allows us to redefine them in our type.

Next up we define the src prop as a required string value. The & will merge this with the previous type.

Next up is where the magic happens. We're defining a conditional type, where we're saying that when layout is provided with the 'fill' value it will use the first. Otherwise it'll use the second type.

The Exclude utility type is similar to Omit.

Below is the completed example. You'll notice there is also the toInt function in order to catch when width or height is the never type instead of passing it directly into the img component.

I found this pattern whilst exploring NextJS' Image component: 

https://github.com/vercel/next.js/blob/canary/packages/next/client/image.tsx

, I have attempted to simplify the example here for demonstration purposes.

In this post I also touched on Utility Types (Exclude and Omit specifically). If you'd like to learn more about them you can see the Typescript documentation: 

https://www.typescriptlang.org/docs/handbook/utility-types.html

Conditionally required props with Typescript in React

One of the things I like from other languages like PHP is how you can throw and catch specific exceptions. Replicating this in JS/TS however requires a little extra work.

Now over in JS/TS environments the try/catch block doesn't support handling different instances of errors. Instead we'd need to add if/else or switch conditions inside the body of the catch in order to handle different cases.

However, in Typescript you cannot simply extend Error and expect to be able to use: if (error instanceof MyError) { }... Since Typescript 2.1 extending built-ins like Error, Array and Map may not work as you'd expect.

The workaround for this is to call: Object.setPrototypeOf(this, MyError.prototype) from within your consructor. Like so:

Previously you would have received an instance of Error instead of MyError.

Typescript handles generating the constructor

This approach to handling errors can produce a bit of nesting in your catch block, however it allows to handle each error differently. I find this useful when you're handling multiple scenarios where an error could be thrown, in some instances you may wish to ignore it or hide it.

The above example shows us how we can better control error messages that could be displayed to an end user. Furthermore I'm still exposing the same API you'd expect from Error. Axios in this case adds the response property where we can access the response from the call that was made, if our code could throw two different errors we'd have the muddy our application logic with checks if certain properties exist. However, I've tucked this away within a function that can be used for making those api calls. Our client that is rendering the error only needs to care about accessing error.message.

Handling custom error classes in Typescript

developing cloud functions with typescript locally

. This post is a follow where I've tackled a few other issues you might have when developing multiple functions from a single repo.

For example, as dependencies grow you won't want to include all dependencies for all functions as they may not make use of them. Large dependency lists will slow down the deployment of your functions, and can impact cold start times. I've addressed this using yarn workspaces and following a mono-repo pattern that is common when developing node packages.

This approach means we can develop shared libraries that can be packaged and deployed to a private NPM registry. These can then be be included as dependencies on your functions. Whilst developing locally you won't notice much of a difference.

By using this pattern each function can be packaged with it's own package.json and deployed separately. This brought up a new problem. My previous solution used a src/index.ts file which imported and exported all functions and was also used as an entrypoint for all requests when developing locally.

To solve this, I have introduced Docker and Docker Compose. Now each function runs in a standalone container. Each container is mapped to a single nginx container which is how you can trigger your functions. I find this approach a lot nicer. Introducing more technologies into this does complicate a bit though, but I believe this has created a more accurate environment.

First up, we'll need to setup our initial repo structure like so:

To create those files form the command line:

So let's quickly talk about this structure. I've tried to keep a logical structure, so all functions are in the src/functions/{event-type}/{function-name}/index.ts format. Supported event-types include: http and event. This just determines which function signature is used (req, res) => {} for http, and (event, context) => {} for event.

The packages directory is where you can add specific packages that will act as your shared code library. Each directory within here must have it's own package.json file which a unique package name. These packages will be scoped, but we'll touch on that more later.

Finally the docker and any docker named files all relate to how we'll build and run our docker containers locally. If you don't currently have docker installed now would be a good time to do so. Docker for Mac is recommended if you're on a Mac. Docker for Windows for Windows, and you can just install Docker if you're on linux. If you're not familiar with docker, I won't be going into very much detail here, so I'm sorry if I glaze over it!

Currently we have three package.json files: ./package.json, src/functions/http/helloWorld/package.json and src/packages/lib/package.json. To get these to work with Yarn Workspaces we'll need to first of all make sure we're running yarn 2 from within our project root:

(berry is the code name for yarn 2 at the time of writing)

Note that yarn 2 now uses a different binary for each project rather than a global yarn. There are a number of differences in how yarn 2 works.

With that done, let's create our root package.json with the following:

functions

In your src/functions/http/helloWorld/package.json add:

functions-http-hello-name

You'll notice I have duplicated a number of the devDependencies here. This is because each workspace must declare all of their dependencies explicitly.

Then in our src/packages/lib/package.json:

@ashsmith-funcs/lib

With that done we can now run yarn install. You'll notice that there is no longer a node_modules directory. This is expected, and part of the new functionality provided by yarn 2.

Previously I was using gts for base Typescript and linting. However using yarn 2 these seemed to break as they depend on the node_modules directory existing. Instead of working around it, I decided to remove it. So lets setup eslint and typescript from scratch this time (using the same configs as gts though for now).

I won't deep dive into these rules. The only one I want to note is the node/no-unpublished-import rule, where I have explicitly allowed the @google-cloud/functions-framework. This is to prevent errors from attempting to load type definitions.

Now the important part here is the compilerOptions.paths section. Here I have stated that all directories within src/packages will be mapped to @ashsmith-funcs/* so our packages/lib would be referenced in another package as @ashsmith-funcs/lib. You'll notice that this is the same name given in the corresponding package.json. This ensures typescript will look for that directory instead of erroring with an unresolved dependency issue.

Ok, let's get into it with our first http function. In src/functions/http/helloWorld/index.ts lets add:

Now run yarn test and make sure it all works!

To test the function locally right now we could also run:

This is great, but what about reloading when file changes happen, and what about developing multiple functions?

For this we'll be using docker-compose to configure our different containers. Let's start by creating our docker images.

Then our docker/start.sh script will look like this:

The start.sh script will run as the main command in the container. This will use nodemon to watch for changes to our project, and reload the functions-framework command. Note the environment variables used here. I'm following a strict naming convention here where by: src/functions/{event}/{functionName}/index.ts must exist, and must export a function called {functionName}.

With this convention we can reuse this docker image for each of our functions, and it helps with deployment processes too.

With our docker images defined. Let's create the docker-compose.yml file:

This file defines three services (or containers):

function-proxy which is a custom nginx server.

tsc-watch this will watch all ts files and compile when they change.

If the images don't exist, it will build them and then boot them up.

The -d will run in detached mode, so it runs in the background.

The last thing to do is configure your hosts file so that 127.0.0.1 is mapped to functions.local:

Then go ahead and open http://functions.local/helloWorld in your browser.

It's worth noting here the ashsmith/function-proxy image uses jwilder/nginx-proxy docker image as a base. This docker image by default allows you to expose containers via this image onto multiple domains (at least 1 domain per container, no shared domains) for each VIRTUAL_HOST environment variable that is defined against a given container.

I have customised the nginx.tmpl file so that we have a single server directive and a single domain, but it generates unique location directives per container. Instead of using VIRTUAL_HOST I use the FUNCTION environment variable to achieve this.

That is how our function helloWorld will be available on: /helloWorld. Nice!

Now we have a function working. Let's create some shared code...

Ok.. it's kind of useless, but provides a useful example!

Then add it to your functions package.json:

Now reload the browser, hopefully the page loads successfully. You can also check the logs output from the docker-compose up command. To see if our console.log made it. If you used the -d parameter then try: docker-compose logs -f hello-world to attach and follow the logs for the helloWorld function.

This approach makes managing dependencies on a per function basis possible. I highly doubt this is the last iteration of the cloud function project for me.

This might be overkill for smaller projects, but it's worth keeping in mind if you wish to keep cold start times down and keep deployments lean.

Combined with the terraform deployment process you'll have a capable environment that has best practices at its core.

With this we also have developed some packages that will need to be published to a private NPM repository. I'll cover this step in a future post! As well as how we will need to modify the deployment strategy (if you've seen my post on 

using terraform to deploy cloud functions

 then that will serve as a reasonable starting point.

Improved mono-repo for Typescript + Cloud Function development

In this post my goal is to show you how to provision and deploy your GCP Cloud Functions by using Terraform. Infrastructure as Code is a great way to define and keep track of all cloud services you put together. My favourite reasons for IaC is it opens up the ability for peer review, and to ability to have the exact same infrastructure defined in multiple environment without issue.

This post follows on from my previous on 

developing GCP Cloud Functions locally with TypeScript

. It'll be useful to have the same project setup if you're following along, if you want to skip that post you can also 

 and be sure to use the developing-cloud-functions branch (trunk includes all the code referenced in this post!).

We'll start by installing Terraform. You can use Homebrew, or download a binary from the Terraform website. However, you may overtime find you're running multiple projects on different versions. For this I recommend 

. This can also be installed via homebrew if you're on MacOS with: brew install tfenv. Check the repo out for additional installation instructions.

Next up let's create the directories and files where we'll be keeping our infrastructure:

There are a fair few files here. It's worth stating now that terraform is fairly free-form, all files are merged together ultimately so you can organise these as you please.

I have defined three core files: functions.tf, providers.tf and variables.tf.

functions.tf is where we'll define our cloud function, along with the storage bucket (for deployment) and the service account the function will use.

providers.tf this is where we will configure settings for our terraform providers. This allows us to set the GCP project id for the google provider. In here I have also locked terraform to specific version to prevent people using a different one (to avoid unexpected breaking changes).

variables.tf is where we will define global variables.

Next up we have the projects directory containing two files: production.tfvars and test.tfvars. This is where we will define the values of global variables. When we execute terraform we can tell it to use correct tfvars file, so we can have specific settings per project (and change to different gcp projects more importantly!).

Finally we have the tf file. This will become an executable that will reduce the verbosity of some the commands we need to run, and helps idiot proof it slightly.

Let's get into it. Starting with our tfvars.

Editing: infra/projects/production.tfvars

Now we need to define this variable in our infra/variables.tf file:

So, what we've done above is define two variables, one called environment the other project. environment represents whether we're running in production or another env. This is handy if you want to deploy multiple versions of the same function to the same GCP project. Additionally we have the project variable which defines our GCP Project ID. You could make this different between production and test to truly separate your architecture.

Now let's configure our infra/providers.tf file:

In this file we have defined a local variable for region this configures both the google-beta and google providers to use the europe-west1 region. Additionally you can see we're referencing the project variable we defined using the var.project syntax. Finally you can also see I have locked the version of terraform down to exactly match 0.13.4 (latest at the time of writing).

Ok, let's move onto the beefy file... infra/functions.tf.

Now, there is a lot going on here. So let's break the steps:

First of all we define a Cloud Storage Bucket. This is where we will upload our packaged function code.

We define the storage object, this will handle uploading our zip that contains the function code. Note that it references function.zip. We'll make this zip later on (outside of terraform)!

We define our Cloud Function. We're referencing where the function is stored, and the service account we create for it. Additionally we have defined the function to have a http trigger.

Next up we've made sure the function can be executed publicly. By default you won't be able execute the function! Be mindful for your application if you actually need this.

Then the next 3 sections define a service account, and then associate the logging.logWriter and monitoring.metricWriter roles to the service account. This SA is assigned to our function, and therefore our function will not be able to interact with any other services other than the above mentioned.

That's quite a lot to take in. I'll break down the terraform syntax below. It's really taken some time to get use to this myself...

Let's go over one of the resources defined line by line to explain:

Hopefully that gives you a little guidance on how terraform resources are defined. As I said before, the docs are the absolute best place to go!

Next up let's set our infra/tf file, this will be need to be executable:

What we're doing here initialising terraform and installing the providers. Then the second command is variable it could run terraform apply, terraform plan or terraform destroy. The $VERB variable is the first argument we accept from the tf script.

Then we define which variable file will be loaded with the -var-file flag. This consumes our second argument PROJECT which could be in this case test or production. Finally the -state option allows us to define where the terraform state will be stored, and we'll keep a two different state files based on the project.

When using multiple state files like this you have to ensure all your resources are defined with this in mind. Terraform will not know if a resource already exists and will attempt to recreate. This is why we prefix our terraform resources where possible with the current environment.

Note: The Terraform state file shouldn't be kept on your local machine. It can contain sensitive values, and if you lose the state file you will have to recreate your infrastructure. In a future post I'll cover managing your state using a remote storage option: 

https://www.terraform.io/docs/state/remote.html

With this all in place, let's modify our package.json to include the commands for running terraform:

That's a lot of commands... However the goal here is to make it obvious which commands are being run. When you run yarn tf:deploy:test it will compile your code, and format your terraform code (they have their own standard format), next up we'll package the build dir and package.json + yarn.lock files into a zip. Then finally we run ./infra/tf apply production this will instruct terraform to create the resources.

Additionally we have an option for tf:plan:{ENV} too, this will show a diff of what will change if you run apply. This can be handy for sanity checking and testing changes without applying.

With a little luck you'll be able to run yarn tf:deploy:test and you'll have your newly provisioned infrastructure in no time!

Note: I have assumed you have setup a GCP Project, linked to a billing account. You will also need to configure the 

Provisioning GCP Cloud Functions with Terraform

In this post I'll show you how I develop Cloud Functions locally using Typescript! I assume you already have some familiarity with Typescript, so we'll dive straight in with getting our project setup.

For this we're going to use Google's typescript style guide tool, 

gts provides us with an no fuss, opinionated, TypeScript setup. It includes a style-guide with linting and auto-fixing. For me this just gets rid of the pain of configuring new projects! gts is extendable, so if you disagree with the style guide then you're more than welcome to customise away to suit your team / project.

Ash Smith

Cloud & Serverless Developer

Blog