One of the things I like from other languages like PHP is how you can throw and catch specific exceptions. Replicating this in JS/TS however requires a little extra work.

Here's a PHP example for reference:

```php
<?php
class MyException extends Exception { }

function myFunction() {
  throw new MyException('some reason');
}

try {
  myFunction();
} catch (MyException $e) {
  // Handle MyException specifically...
} catch (Exception $e) {
  // Handle all other exceptions
}
```

Now over in JS/TS environments the `try/catch` block doesn't support handling different instances of errors. Instead we'd need to add `if`/`else` or `switch` conditions inside the body of the `catch` in order to handle different cases.

However, in Typescript you cannot simply extend `Error` and expect to be able to use: `if (error instanceof MyError) { }`... Since Typescript 2.1 extending built-ins like `Error`, `Array` and `Map` may not work as you'd expect.

The workaround for this is to call: `Object.setPrototypeOf(this, MyError.prototype)` from within your consructor. Like so:

```ts
class MyError extends Error {
  constructor(message: string) {
    super(message);
    Object.setPrototypeOf(this, MyError.prototype)
  }
}
```

Now the following code will work:

```ts
const myFunction = () => {
  throw new MyError('some reason');
}

try {
  myFunction();
} catch (error) {
  if (error instanceof MyError) {
    // Handle MyError....
  }
}
```

Previously you would have received an instance of `Error` instead of `MyError`.

This is due to how [Typescript handles generating the constructor](https://github.com/Microsoft/TypeScript/wiki/Breaking-Changes#generated-constructor-code-substitutes-the-return-value-of-super-calls-as-this).

This approach to handling errors can produce a bit of nesting in your `catch` block, however it allows to handle each error differently. I find this useful when you're handling multiple scenarios where an error could be thrown, in some instances you may wish to ignore it or hide it.

Consider the following:

```ts
import axios from 'axios';

class JSONAPIError extends Error {
  constructor(message: string) {
    super(message);
    Object.setPrototypeOf(this, MyError.prototype)
  }

  public static fromAxiosError(error: AxiosError) {
    // Surface the detailed error result from our JSON:API compliant api.
    // This error could be more useful to end users
    // Axios error.message contains the status and statusText which
    // isnt useful to users.
    return new JSONAPIError(error.response?.data?.errors[0].detail ?? error.message);
  }
}

const makeApiCall = async (method, url) => {
  try {
    const response = await axios.request({ method, url });
  } catch(error) {
    if (error.response.status === 500) {
      // Notify your error handling system.
      console.error(error);
      // Throw again with a bett
      throw new Error('Internal server error, please try again or contact support');
    }

    throw JSONAPIError.fromAxiosError(error);
  }
}

try {
  const result = await makeApiCall('GET', 'https://google.com/');
  return result;
} catch(error) {
  // A consistent API for handling errors with half decent messages 😍
  window.alert(error.message);
}

```

The above example shows us how we can better control error messages that could be displayed to an end user. Furthermore I'm still exposing the same API you'd expect from `Error`. Axios in this case adds the `response` property where we can access the response from the call that was made, if our code could throw two different errors we'd have the muddy our application logic with checks if certain properties exist. However, I've tucked this away within a function that can be used for making those api calls. Our client that is rendering the error only needs to care about accessing `error.message`. 

Handling custom error classes in Typescript

I recently posted about [developing cloud functions with typescript locally](https://www.ashsmith.io/developing-gcp-cloud-functions-locally-with-typescript). This post is a follow where I've tackled a few other issues you might have when developing multiple functions from a single repo.

For example, as dependencies grow you won't want to include all dependencies for all functions as they may not make use of them. Large dependency lists will slow down the deployment of your functions, and can impact cold start times. I've addressed this using yarn workspaces and following a mono-repo pattern that is common when developing node packages.

This approach means we can develop shared libraries that can be packaged and deployed to a private NPM registry. These can then be be included as dependencies on your functions. Whilst developing locally you won't notice much of a difference.

By using this pattern each function can be packaged with it's own package.json and deployed separately. This brought up a new problem. My previous solution used a `src/index.ts` file which imported and exported all functions and was also used as an entrypoint for all requests when developing locally.

To solve this, I have introduced Docker and Docker Compose. Now each function runs in a standalone container. Each container is mapped to a single nginx container which is how you can trigger your functions. I find this approach a lot nicer. Introducing more technologies into this does complicate a bit though, but I believe this has created a more accurate environment.

Let's get started.

## Initial repo structure:
First up, we'll need to setup our initial repo structure like so:
```
docker/
  functions/
    Dockerfile
    start.sh
src/
  functions/
    http/
      helloWorld/
        index.ts
        index.test.ts
        package.json
  packages/
    lib/
      index.ts
      package.json
.eslintrc.json
.gitignore
docker-compose.yml
package.json
tsconfig.json
```

To create those files form the command line:

```bash
mkdir -p docker/functions
touch docker/functions/{Dockerfile,start.sh}
mkdir -p src/{functions/packages}
mkdir -p src/functions/{event,http}
mkdir -p src/functions/http/helloWorld/{index.ts,package.json}
mkdir -p src/packages/lib/{index.ts,package.json}
touch .eslintrc.json docker-compose.yml package.json tsconfig.json
```

So let's quickly talk about this structure. I've tried to keep a logical structure, so all functions are in the `src/functions/{event-type}/{function-name}/index.ts` format. Supported event-types include: `http` and `event`. This just determines which function signature is used `(req, res) => {}` for http, and `(event, context) => {}` for event.

The `packages` directory is where you can add specific packages that will act as your shared code library. Each directory within here must have it's own `package.json` file which a unique package name. These packages will be scoped, but we'll touch on that more later.

Finally the `docker` and any docker named files all relate to how we'll build and run our docker containers locally. If you don't currently have docker installed now would be a good time to do so. Docker for Mac is recommended if you're on a Mac. Docker for Windows for Windows, and you can just install Docker if you're on linux. If you're not familiar with docker, I won't be going into very much detail here, so I'm sorry if I glaze over it!

## Setting up our package.json files.

Currently we have three `package.json` files: `./package.json`, `src/functions/http/helloWorld/package.json` and `src/packages/lib/package.json`. To get these to work with Yarn Workspaces we'll need to first of all make sure we're running yarn 2 from within our project root:

```bash
yarn set version berry
```
(berry is the code name for yarn 2 at the time of writing)

Note that yarn 2 now uses a different binary for each project rather than a global yarn. There are a number of differences in how yarn 2 works.

With that done, let's create our root package.json with the following:
```json
{
  "name": "functions",
  "version": "0.0.0",
  "description": "",
  "license": "Apache-2.0",
  "keywords": [],
  "private": true,
  "workspaces": [
    "src/packages/*",
    "src/functions/http/*",
    "src/functions/event/*"
  ],
  "scripts": {
    "dev": "yarn install && docker-compose up",
    "test": "mocha -r ts-node/register ./**/*.test.ts",
    "lint": "eslint '**/*.ts'",
    "compile": "tsc",
    "fix": "eslint --fix '**/*.ts'"
  },
  "devDependencies": {
    "@google-cloud/functions-framework": "^1.7.1",
    "@types/chai": "^4.2.14",
    "@types/eslint": "^7.2.4",
    "@types/express": "^4.17.8",
    "@types/mocha": "^8.0.3",
    "@types/node": "^14.11.2",
    "@types/prettier": "^2.1.5",
    "@typescript-eslint/eslint-plugin": "^4.5.0",
    "@typescript-eslint/parser": "^4.5.0",
    "chai": "^4.2.0",
    "chai-http": "^4.3.0",
    "eslint": "^7.11.0",
    "eslint-config-prettier": "^6.13.0",
    "eslint-plugin-node": "^11.1.0",
    "eslint-plugin-prettier": "^3.1.4",
    "express": "^4.17.1",
    "mocha": "^8.2.0",
    "nodemon": "^2.0.4",
    "prettier": "^2.1.2",
    "ts-node": "^9.0.0",
    "typescript": "^4.0.3"
  }
}
```

In your `src/functions/http/helloWorld/package.json` add:

```json
{
  "name": "functions-http-hello-name",
  "version": "0.0.0",
  "description": "",
  "license": "Apache-2.0",
  "keywords": [],
  "main": "build/index.js",
  "rootDir": ".",
  "scripts": {
    "start": "functions-framework",
    "test": "mocha -r ts-node/register ./**/*.test.ts",
    "build": "tsc",
    "tsc:watch": "tsc --watch",
    "prezip": "cp package.json build",
    "zip": "cd build && zip -r ../function.zip ."
  },
  "devDependencies": {
    "@google-cloud/functions-framework": "^1.7.1",
    "@types/chai": "^4.2.14",
    "@types/eslint": "^7.2.4",
    "@types/express": "^4.17.8",
    "@types/mocha": "^8.0.3",
    "chai": "^4.2.0",
    "chai-http": "^4.3.0",
    "eslint": "^7.11.0",
    "express": "^4.17.1",
    "mocha": "^8.2.0",
    "typescript": "^4.0.3"
  }
}
```

You'll notice I have duplicated a number of the devDependencies here. This is because each workspace must declare all of their dependencies explicitly.

Then in our `src/packages/lib/package.json`:
```json
{
  "name": "@ashsmith-funcs/lib",
  "version": "1.0.0",
  "license": "MIT",
  "main": "build/index.js",
  "files": [
    "build/**/*"
  ],
  "types": "build/index.d.ts",
  "scripts": {
    "build": "tsc",
    "tsc:watch": "tsc --watch"
  },
  "devDependencies": {
    "@types/chai": "^4.2.14",
    "@types/eslint": "^7.2.4",
    "@types/express": "^4.17.8",
    "@types/mocha": "^8.0.3",
    "chai": "^4.2.0",
    "chai-http": "^4.3.0",
    "eslint": "^7.11.0",
    "express": "^4.17.1",
    "mocha": "^8.2.0",
    "typescript": "^4.0.3"
  }
}
```

With that done we can now run `yarn install`. You'll notice that there is no longer a `node_modules` directory. This is expected, and part of the new functionality provided by yarn 2.

## Setting up Typescript and linting

Previously I was using `gts` for base Typescript and linting. However using yarn 2 these seemed to break as they depend on the node_modules directory existing. Instead of working around it, I decided to remove it. So lets setup eslint and typescript from scratch this time (using the same configs as gts though for now).

Your `eslintrc.json` will look like this:
```json
{
  "extends": [
    "eslint:recommended",
    "plugin:node/recommended",
    "prettier"
  ],
  "plugins": [
    "node",
    "prettier"
  ],
  "rules": {
    "prettier/prettier": "error",
    "block-scoped-var": "error",
    "eqeqeq": "error",
    "no-var": "error",
    "prefer-const": "error",
    "eol-last": "error",
    "prefer-arrow-callback": "error",
    "no-trailing-spaces": "error",
    "quotes": ["warn", "single", { "avoidEscape": true }],
    "no-restricted-properties": [
      "error",
      {
        "object": "describe",
        "property": "only"
      },
      {
        "object": "it",
        "property": "only"
      }
    ],
    "node/no-unpublished-import": ["error", {
      "allowModules": ["@google-cloud/functions-framework"]
    }]
  },
  "overrides": [
    {
      "files": ["**/*.ts", "**/*.tsx"],
      "parser": "@typescript-eslint/parser",
      "extends": [
        "plugin:@typescript-eslint/recommended"
      ],
      "rules": {
        "@typescript-eslint/no-non-null-assertion": "off",
        "@typescript-eslint/no-use-before-define": "off",
        "@typescript-eslint/no-warning-comments": "off",
        "@typescript-eslint/no-empty-function": "off",
        "@typescript-eslint/no-var-requires": "off",
        "@typescript-eslint/explicit-function-return-type": "off",
        "@typescript-eslint/explicit-module-boundary-types": "off",
        "@typescript-eslint/ban-types": "off",
        "@typescript-eslint/camelcase": "off",
        "node/no-missing-import": "off",
        "node/no-empty-function": "off",
        "node/no-unsupported-features/es-syntax": "off",
        "node/no-missing-require": "off",
        "node/shebang": "off",
        "no-dupe-class-members": "off",
        "require-atomic-updates": "off"
      },
      "parserOptions": {
        "ecmaVersion": 2018,
        "sourceType": "module"
      }
    }
  ]
}
```

I won't deep dive into these rules. The only one I want to note is the `node/no-unpublished-import` rule, where I have explicitly allowed the `@google-cloud/functions-framework`. This is to prevent errors from attempting to load type definitions.

`prettier.json`
```json
{
  "bracketSpacing": false,
  "singleQuote": true,
  "trailingComma": "es5",
  "arrowParens": "avoid"
}
```

Next up, our `tsconfig.json`
```json
{
  "compilerOptions": {
    "lib": ["es2018", "dom"],
    "allowUnreachableCode": false,
    "allowUnusedLabels": false,
    "declaration": true,
    "forceConsistentCasingInFileNames": true,
    "module": "commonjs",
    "noEmitOnError": true,
    "noFallthroughCasesInSwitch": true,
    "noImplicitReturns": true,
    "pretty": true,
    "sourceMap": true,
    "strict": true,
    "target": "es2018",

    "esModuleInterop": true,
    "baseUrl": "src",
    "paths": {
        "@ashsmith-funcs/*": [
          "packages/*"
        ],
    }
  },
  "exclude": ["node_modules", "./**/*.test.ts"]
}
```

Now the important part here is the `compilerOptions.paths` section. Here I have stated that all directories within `src/packages` will be mapped to `@ashsmith-funcs/*` so our packages/lib would be referenced in another package as `@ashsmith-funcs/lib`. You'll notice that this is the same name given in the corresponding package.json. This ensures typescript will look for that directory instead of erroring with an unresolved dependency issue.

## Setting up our first function

Ok, let's get into it with our first http function. In `src/functions/http/helloWorld/index.ts` lets add:

```typescript
import type {HttpFunction} from '@google-cloud/functions-framework/build/src/functions';

export const helloWorld: HttpFunction = (req, res) => {
  res.send('Hello, World!');
};
```

In our test file `index.test.ts`
```typescript
/* eslint-disable node/no-extraneous-import */
import chai from 'chai';
import chaiHttp from 'chai-http';
import express from 'express';
import {helloName} from './';

chai.use(chaiHttp);
chai.should();

const app = express();
app.all('*', helloName);

describe('helloName function', () => {
  describe('GET /', () => {
    it('should get return Hello, World! when no query is provided ', done => {
      chai
        .request(app)
        .get('/')
        .end((err, res) => {
          res.should.have.status(200);
          res.text.should.be.a('string');
          res.text.should.equal('Hello, World!');
          done();
        });
    });
    it('should get return Hello, ash! when the query string is added: ?name=ash', done => {
      chai
        .request(app)
        .get('/?name=ash')
        .end((err, res) => {
          res.should.have.status(200);
          res.text.should.be.a('string');
          res.text.should.equal('Hello, ash!');
          done();
        });
    });
  });
});

```

Now run `yarn test` and make sure it all works!

To test the function locally right now we could also run:
```bash
yarn workspaces functions-http-hello-world run start --target=helloWorld
```

This will start up localhost:8080.

This is great, but what about reloading when file changes happen, and what about developing multiple functions?

Lets dive into docker.

## Docker setup.

For this we'll be using docker-compose to configure our different containers. Let's start by creating our docker images.

`docker/functions/Dockerfile`:
```dockerfile
FROM node:12

COPY ./start.sh /usr/bin/start.sh
RUN chmod +x /usr/bin/start.sh

EXPOSE 8080

USER node
WORKDIR /home/node

RUN yarn set version berry

CMD /usr/bin/start.sh

```

Then our `docker/start.sh` script will look like this:
```bash
#!/bin/bash

npx nodemon \
  --watch src/functions/${EVENT}/${FUNCTION} \
  --exec yarn workspace ${WORKSPACE} run start \
    --signature_type=${EVENT} \
    --source=./build \
    --target=${FUNCTION}

```

The `start.sh` script will run as the main command in the container. This will use nodemon to watch for changes to our project, and reload the functions-framework command. Note the environment variables used here. I'm following a strict naming convention here where by: `src/functions/{event}/{functionName}/index.ts` must exist, and must export a function called `{functionName}`.

With this convention we can reuse this docker image for each of our functions, and it helps with deployment processes too.

With our docker images defined. Let's create the `docker-compose.yml` file:

```yaml
version: '2'

services:
  function-proxy:
    image: ashsmith/function-proxy
    ports:
      - "80:80"
    volumes:
      - /var/run/docker.sock:/tmp/docker.sock:ro
    environment:
      - DEFAULT_HOST=functions.local

  tsc-watch:
    image: ashsmith/functions-framework
    build:
      context: ./docker/functions
    working_dir: /home/node
    volumes:
      - ./:/home/node
    command: ["yarn", "workspaces", "foreach", "-ipa", "run", "tsc:watch"]

  hello-world:
    image: ashsmith/functions-framework
    build:
      context: ./docker/functions
    volumes:
      - ./:/home/node
    environment:
      - FUNCTION=helloWorld
      - EVENT=http
      - WORKSPACE=functions-http-hello-world

```

This file defines three services (or containers):

- `function-proxy` which is a custom nginx server.
- `tsc-watch` this will watch all ts files and compile when they change.
- `hello-world` which is our function.

To get this running:
```bash
docker-compose up [-d]
```

If the images don't exist, it will build them and then boot them up.

The `-d` will run in detached mode, so it runs in the background.

The last thing to do is configure your hosts file so that `127.0.0.1` is mapped to `functions.local`:

On mac / linux:

```bash
echo "127.0.0.1 functions.local" | sudo tee -a /etc/hosts
```

Then go ahead and open `http://functions.local/helloWorld` in your browser.

It's worth noting here the ashsmith/function-proxy image uses jwilder/nginx-proxy docker image as a base. This docker image by default allows you to expose containers via this image onto multiple domains (at least 1 domain per container, no shared domains) for each `VIRTUAL_HOST` environment variable that is defined against a given container.

I have customised the nginx.tmpl file so that we have a single server directive and a single domain, but it generates unique `location` directives per container. Instead of using `VIRTUAL_HOST` I use the `FUNCTION` environment variable to achieve this.

That is how our function `helloWorld` will be available on: `/helloWorld`. Nice!

## Adding a dependency...

Now we have a function working. Let's create some shared code...

`src/packages/lib/index.ts`

```typescript
class Logger {
  public static info(...params: any[]) {
    console.info(...params);
  }
  public static warn(...params: any[]) {
    console.warn(...params);
  }
  public static error(...params: any[]) {
    console.error(...params);
  }
}
export { Logger };

```

Ok.. it's kind of useless, but provides a useful example!

Now let's modify our function to use it:

`src/functions/http/helloWorld/index.ts`

```typescript
import type {HttpFunction} from '@google-cloud/functions-framework/build/src/functions';
import {Logger} from '@ashsmith-funcs/lib';

export const helloWorld: HttpFunction = (req, res) => {
  Logger.info('hello world!');
  res.send('Hello, World!');
};

```

Then add it to your functions package.json:
```
yarn workspace functions-http-hello-world add @ashsmith-funcs/lib
```

Now reload the browser, hopefully the page loads successfully. You can also check the logs output from the `docker-compose up` command. To see if our console.log made it. If you used the `-d` parameter then try: `docker-compose logs -f hello-world` to attach and follow the logs for the helloWorld function.

## Final thoughts.

This approach makes managing dependencies on a per function basis possible. I highly doubt this is the last iteration of the cloud function project for me.

This might be overkill for smaller projects, but it's worth keeping in mind if you wish to keep cold start times down and keep deployments lean.

Combined with the terraform deployment process you'll have a capable environment that has best practices at its core.

With this we also have developed some packages that will need to be published to a private NPM repository. I'll cover this step in a future post! As well as how we will need to modify the deployment strategy (if you've seen my post on [using terraform to deploy cloud functions](https://www.ashsmith.io/provisioning-gcp-cloud-functions-with-terraform) then that will serve as a reasonable starting point.


Improved mono-repo for Typescript + Cloud Function development

In this post my goal is to show you how to provision and deploy your GCP Cloud Functions by using Terraform. Infrastructure as Code is a great way to define and keep track of all cloud services you put together. My favourite reasons for IaC is it opens up the ability for peer review, and to ability to have the exact same infrastructure defined in multiple environment without issue.

This post follows on from my previous on [developing GCP Cloud Functions locally with TypeScript](https://www.ashsmith.io/developing-gcp-cloud-functions-locally-with-typescript). It'll be useful to have the same project setup if you're following along, if you want to skip that post you can also [clone down this repo](https://github.com/ashsmith/local-dev-gcp-cloud-functions/tree/developing-cloud-functions) and be sure to use the `developing-cloud-functions` branch (`trunk` includes all the code referenced in this post!).

We'll start by installing Terraform. You can use Homebrew, or download a binary from the Terraform website. However, you may overtime find you're running multiple projects on different versions. For this I recommend [tfenv](https://github.com/tfutils/tfenv). This can also be installed via homebrew if you're on MacOS with: `brew install tfenv`. Check the repo out for additional installation instructions.

Next up let's create the directories and files where we'll be keeping our infrastructure:

```bash
mkdir infra/projects
touch infra/{functions,providers,variables}.tf infra/tf infra/project/{production,test}.tfvars
```

There are a fair few files here. It's worth stating now that terraform is fairly free-form, all files are merged together ultimately so you can organise these as you please.

I have defined three core files: `functions.tf`, `providers.tf` and `variables.tf`.

`functions.tf` is where we'll define our cloud function, along with the storage bucket (for deployment) and the service account the function will use.

`providers.tf` this is where we will configure settings for our terraform providers. This allows us to set the GCP project id for the google provider. In here I have also locked terraform to specific version to prevent people using a different one (to avoid unexpected breaking changes).

`variables.tf` is where we will define global variables.

Next up we have the `projects` directory containing two files: `production.tfvars` and `test.tfvars`. This is where we will define the values of global variables. When we execute terraform we can tell it to use correct tfvars file, so we can have specific settings per project (and change to different gcp projects more importantly!).

Finally we have the `tf` file. This will become an executable that will reduce the verbosity of some the commands we need to run, and helps idiot proof it slightly.

Let's get into it. Starting with our tfvars.

Editing: `infra/projects/production.tfvars`
```
environment = "prod"
project     = "[YOUR-GCP-PROJECT-ID-HERE]"
```

Editing: `infra/projects/test.tfvars`
```
environment = "test"
project     = "[YOUR-GCP-PROJECT-ID-HERE]"
```

Now we need to define this variable in our `infra/variables.tf` file:
```
variable "environment" {}
variable "project" {}
```

So, what we've done above is define two variables, one called `environment` the other `project`. `environment` represents whether we're running in production or another env. This is handy if you want to deploy multiple versions of the same function to the same GCP project. Additionally we have the `project` variable which defines our GCP Project ID. You could make this different between `production` and `test` to truly separate your architecture.

Now let's configure our `infra/providers.tf` file:
```
locals {
  region = "europe-west1"
}

provider "google-beta" {
  project = var.project
  region  = local.region
}

provider "google" {
  project = var.project
  region  = local.region
}

terraform {
  required_version = "0.13.4"
}
```

In this file we have defined a local variable for `region` this configures both the `google-beta` and `google` providers to use the `europe-west1` region. Additionally you can see we're referencing the `project` variable we defined using the `var.project` syntax. Finally you can also see I have locked the version of terraform down to exactly match 0.13.4 (latest at the time of writing).

Ok, let's move onto the beefy file... `infra/functions.tf`.

```
# GCS bucket for storing our uploaded function zip.
# @see https://www.terraform.io/docs/providers/google/r/storage_bucket.html
resource "google_storage_bucket" "function_artifacts" {
  name = "${var.project}-function-artifacts"
}

# GCS Object for our hello_world function zip.
# @see https://www.terraform.io/docs/providers/google/r/storage_bucket_object.html
resource "google_storage_bucket_object" "hello_world_gcf" {
  name   = "${var.environment}_hello_world_gcf/${timestamp()}.zip"
  bucket = google_storage_bucket.function_artifacts.name
  source = "functions.zip"
}

# Define our function
# @see https://www.terraform.io/docs/providers/google/r/cloudfunctions_function.html
resource "google_cloudfunctions_function" "hello_world" {
  # Make the name unique per environment!
  name    = "${var.environment}-helloWorld"
  runtime = "nodejs10"

  # The exported function we wish to execute from within build/src/index.js
  entry_point = "helloWorld"

  source_archive_bucket = google_storage_bucket.function_artifacts.id
  source_archive_object = google_storage_bucket_object.hello_world_gcf.output_name

  # Our service account that only allows this function to write to logs and metrics.
  service_account_email = google_service_account.hello_world_gcf.email

  # Allow our function to be triggered via http requests.
  trigger_http = true

}

# Allow our cloud function to invoked by all users - WARNING this make your cloud function public.
# be sure to check the docs and consider if this is really what you want.
# @see https://www.terraform.io/docs/providers/google/r/cloudfunctions_cloud_function_iam.html
resource "google_cloudfunctions_function_iam_member" "hello_world" {
  cloud_function = google_cloudfunctions_function.hello_world.name
  role           = "roles/cloudfunctions.invoker"
  member         = "allUsers"
}

# A service account just for our helloWorld function.
# this ensures our cloud function only has the absolute minimum needed permissions.
# @see https://www.terraform.io/docs/providers/google/r/google_service_account.html
resource "google_service_account" "hello_world_gcf" {
  provider     = google-beta
  account_id   = "${var.environment}-gcf-helloworld"
  display_name = "${var.environment}-gcf-helloworld"
}

# Add the monitoring.metricWriter role to our service account.
# @see https://www.terraform.io/docs/providers/google/r/google_project_iam.html#google_project_iam_member
resource "google_project_iam_member" "hello_world_gcf_monitoring_writer" {
  provider = google-beta
  role     = "roles/monitoring.metricWriter"
  member   = "serviceAccount:${google_service_account.hello_world_gcf.email}"
}

# Add the logging.logWriter role to our service account.
# @see https://www.terraform.io/docs/providers/google/r/google_project_iam.html#google_project_iam_member
resource "google_project_iam_member" "hello_world_gcf_logging_writer" {
  provider = google-beta
  role     = "roles/logging.logWriter"
  member   = "serviceAccount:${google_service_account.hello_world_gcf.email}"
}

```

Now, there is a lot going on here. So let's break the steps:

1. First of all we define a Cloud Storage Bucket. This is where we will upload our packaged function code.
2. We define the storage object, this will handle uploading our zip that contains the function code. Note that it references `function.zip`. We'll make this zip later on (outside of terraform)!
3. We define our Cloud Function. We're referencing where the function is stored, and the service account we create for it. Additionally we have defined the function to have a http trigger. 
4. Next up we've made sure the function can be executed publicly. By default you won't be able execute the function! Be mindful for your application if you actually need this.
5. Then the next 3 sections define a service account, and then associate the logging.logWriter and monitoring.metricWriter roles to the service account. This SA is assigned to our function, and therefore our function will not be able to interact with any other services other than the above mentioned.

That's quite a lot to take in. I'll break down the terraform syntax below. It's really taken some time to get use to this myself...

Let's go over one of the resources defined line by line to explain:

```
# First we define a resource with: 
# resource "resource_name_from_provider_docs" "my_unique_name"
# the "my_unique_name" will be stored in the terraform state file, and will map to the resources created within GCP.
resource "google_storage_bucket_object" "hello_world_gcf" {
  # Different resources have different properties. The docs are the best place to find out what these fields do.
  # here we've set a name with some string interpolation, this will create a new object called: test_hello_world_gcf/2020-10-12T21:54:44Z.zip
  name   = "${var.environment}_hello_world_gcf/${timestamp()}.zip"

  # Here we're referencing the resource above, and accessing the name property.
  bucket = google_storage_bucket.function_artifacts.name
  source = "functions.zip"
}
```

Hopefully that gives you a little guidance on how terraform resources are defined. As I said before, the docs are the absolute best place to go!

Next up let's set our `infra/tf` file, this will be need to be executable:

```bash
chmod +x infra/tf
```

Then the contents will look like so:

```
#!/bin/bash

VERB=$1
PROJECT=$2

terraform init -reconfigure infra

terraform $VERB -var-file=./infra/projects/${PROJECT}.tfvars -state=./infra/state/${PROJECT}-terraform.tfstate infra

```

What we're doing here initialising terraform and installing the providers. Then the second command is variable it could run `terraform apply`, `terraform plan` or `terraform destroy`. The `$VERB` variable is the first argument we accept from the `tf` script. 

Then we define which variable file will be loaded with the `-var-file` flag. This consumes our second argument `PROJECT` which could be in this case `test` or `production`. Finally the `-state` option allows us to define where the terraform state will be stored, and we'll keep a two different state files based on the project.

When using multiple state files like this you have to ensure all your resources are defined with this in mind. Terraform will not know if a resource already exists and will attempt to recreate. This is why we prefix our terraform resources where possible with the current environment.

> Note: The Terraform state file shouldn't be kept on your local machine. It can contain sensitive values, and if you lose the state file you will have to recreate your infrastructure. In a future post I'll cover managing your state using a remote storage option: https://www.terraform.io/docs/state/remote.html

With this all in place, let's modify our `package.json` to include the commands for running terraform:

```diff
-   "compile": "tsc",
+   "compile": "tsc && yarn tf:format",
    "fix": "gts fix",
-   "prepare": "yarn compile"
+   "prepare": "yarn compile && zip -r functions.zip build && zip -g functions.zip {package.json,yarn.lock}",
+   "tf:format": "terraform fmt -recursive infra",
+   "tf:plan": "yarn prepare && ./infra/tf plan",
+   "tf:plan:test": "yarn tf:plan test",
+   "tf:plan:prod": "yarn tf:plan production",
+   "tf:deploy": "yarn prepare && ./infra/tf apply",
+   "tf:deploy:test": "yarn tf:deploy test",
+   "tf:deploy:prod": "yarn tf:deploy production",
+   "tf:destroy:test": "./infra/tf destroy test",
+   "tf:destroy:prod": "./infra/tf destroy production"
```

That's a lot of commands... However the goal here is to make it obvious which commands are being run. When you run `yarn tf:deploy:test` it will compile your code, and format your terraform code (they have their own standard format), next up we'll package the build dir and package.json + yarn.lock files into a zip. Then finally we run `./infra/tf apply production` this will instruct terraform to create the resources.

Additionally we have an option for `tf:plan:{ENV}` too, this will show a diff of what will change if you run apply. This can be handy for sanity checking and testing changes without applying.

With a little luck you'll be able to run `yarn tf:deploy:test` and you'll have your newly provisioned infrastructure in no time!

> Note: I have assumed you have setup a GCP Project, linked to a billing account.
> You will also need to configure the [gcloud cli and authenticated](https://cloud.google.com/sdk/docs/install)
> You may also need to enable the [Cloud Build API](https://console.developers.google.com/apis/api/cloudbuild.googleapis.com/overview)

Provisioning GCP Cloud Functions with Terraform

In this post I'll show you how I develop Cloud Functions locally using Typescript! I assume you already have some familiarity with Typescript, so we'll dive straight in with getting our project setup.

For this we're going to use Google's typescript style guide tool, [gts](https://github.com/google/gts).

> `gts` provides us with an no fuss, opinionated, TypeScript setup. It includes a style-guide with linting and auto-fixing. For me this just gets rid of the pain of configuring new projects! `gts` is extendable, so if you disagree with the style guide then you're more than welcome to customise away to suit your team / project.

```bash
mkdir my-cf-project
cd my-cf-project
# use --yarn if you want to use yarn as your package manager, omit for npm.
npx gts init -y --yarn
```

If you run a quick `ls` of the current directory you'll see that `gts` has setup your TS project. Excellent!

> Note: You can use the `yarn lint` and `yarn fix` commands to lint and automatically fix most formatting problems in your code. You can also configure your IDE to do this too.

Next up, we need a way of running Cloud Functions locally. For this we can use Google's Functions Framework. Over on their github you'll see various run times for NodeJS, PHP, Python and more. For us, we're using their NodeJS library.

We can add that as a dev dependency, along with the `@types/express` to include the express packages TypeScript definitions (without this you may get an error from `tsc` saying it couldn't find the type definitions).

```bash
yarn add -D @google-cloud/functions-framework @types/express
# or if using npm:
npm install --save-dev @google-cloud/functions-framework @types/express
```

In addition to this we'll need the following packages too: `concurrently` and `nodemon`. This will allow us to use nodemon to restart our functions framework CLI tool whenever files change, and continuously watch and recompile our TypeScript code.

```bash
yarn add -D concurrently nodemon
# or if using npm:
npm install --save-dev concurrently nodemon
```

Then we'll need to modify our `package.json` to add our new scripts:

```json
{
  ...
  "scripts": {
    "start": "functions-framework --source=build/src/ --target=helloWorld",
    "watch": "concurrently \"tsc -w\" \"nodemon --watch ./build/ --exec npm run start\"",
    ...
  },
}
```

So here our `start` script will define the `build/src` directory as the source for our function and then `--target=helloWorld` defines our function which will be executed when requests come into the functions framework (via http requests).

Next up, lets create replace `src/index.ts` with the following:

```typescript
import type {HttpFunction} from '@google-cloud/functions-framework/build/src/functions';

export const helloWorld: HttpFunction = (req, res) => {
  res.send('Hello, World!');
};
```

With that done, we're ready to test! You can use the `watch` command to compile and run your application for development.

```bash
yarn watch
# or:
npm watch
```

And, with a little luck, you can open up http://localhost:8080/ and you'll see `Hello, World!` in your browser!

### But wait, what about developing multiple functions!?

This solution is great if you want one repo per function, or to manage each function with it's own configuration per directory. Over time however this will become more cumbersome to manage. So let's refactor this a little and get it working for multiple functions.

It's worth saying that at the time of writing Google's Function Framework does not currently support developing multiple functions. Therefore, we need a little hack.

The best way I've found is to have a single entrypoint function, which then based on the URL path will route through to your individual functions. This works perfectly for any HTTP functions you have. I'll cover other functions in another post.

So with that in mind, rename your `src/index.ts` to `src/helloWorld.ts` and create a new `src/index.ts`

In the newly created `src/index.ts` you'll want to receive the request then route it through to your `helloWorld.ts` function. Like so:

```typescript
import type {HttpFunction} from '@google-cloud/functions-framework/build/src/functions';
import {helloWorld} from './helloWorld';

const ROUTES = [
  {
    pathRegex: /^\/helloWorld/,
    function: helloWorld,
  },
];

export const index: HttpFunction = (req, res) => {
  const routes = ROUTES.filter(route => req.path.match(route.pathRegex));
  if (routes.length === 0) {
    throw new Error(
      'Unknown path, have you defined your function in src/index.ts?'
    );
  }

  const [entrypoint] = routes;
  return entrypoint.function(req, res);
};

export {helloWorld};

```

> You may noticed the last line: `export {helloWorld};`, why am I doing this? Well GCP Cloud Functions don't allow you specify which file and which function to execute. By default it will look for either `index.js` or `function.js`. Therefore to deploy any of our functions they will need to be exported from our `index.ts` file too.

You'll then need to modify your `package.json` start script to point to your exported `index` function:

```diff
+   "start": "functions-framework --source=build/src/ --target=index",
-   "start": "functions-framework --source=build/src/ --target=helloWorld",
```

Now with this solution you can add a new object into the `ROUTES` constant any time you wish to add a new function to be triggered, along with a path regex to match. I also throw an error when no routes are matched with a handy message to help guide the developer as to what might be wrong.

It isn't perfect. But we can make additional modifications along the way if we need. For example, you may expect the request path (`req.path`) for your function to be empty (i.e. routing to the index `/` of the cloud function domain). That can be achieved by adding `req.path.replace(entrypoint.pathRegex, '');` before you execute the function on line 20. So our new `index.ts` will look like this:

```diff
  const [entrypoint] = routes;
+ req.path.replace(entrypoint.pathRegex, '');
  return entrypoint.function(req, res);
```

### What about testing?

For testing we'll use mocha, chai and chai-http. However, before we can install these dependencies we will need to make a small change to our `tsconfig.json`. Due to a dependency on `chai-http` that expects some browser based types/interfaces to be defined. So, under the `compilerOptions` object we'll add `"lib": ["es2018", "dom"]`, and we also need to add `"esModuleInterop": true,` so that `chai` and `chai-http` can be imported using es6 import syntax.

```diff
{
  "extends": "./node_modules/gts/tsconfig-google.json",
  "compilerOptions": {
    "rootDir": ".",
    "outDir": "build",
+   "lib": ["es2018", "dom"],
+   "esModuleInterop": true,
  },
  "include": [
    "src/**/*.ts",
    "test/**/*.ts"
  ]
}

```

Next up, let's add our dependencies for testing:

```bash
yarn add -D express ts-node chai chai-http mocha @types/chai @types/mocha
```

Next modify our `package.json` to include our test script:

```diff
-    "test": "echo \"Error: no test specified\" && exit 1",
+    "test": "mocha -r ts-node/register test/**/*.test.ts",
```

Then let's create a simple test for our `helloWorld` function:

```
mkdir test
touch test/helloWorld.test.ts
```
For our `test/helloWorld.test.ts` file:
```typescript
import chai from 'chai';
import chaiHttp from 'chai-http';
import express from 'express';
import {helloWorld} from '../src/helloWorld';

chai.use(chaiHttp);
chai.should();

const app = express();
app.all('/', helloWorld);

describe('helloWorld function', () => {
  describe('GET /', () => {
    it('should get return hello world', done => {
      chai
        .request(app)
        .get('/')
        .end((err, res) => {
          res.should.have.status(200);
          res.text.should.be.a('string');
          res.text.should.equal('Hello, World!');
          done();
        });
    });
  });
});
```

Now you can run `yarn test` and see that our tests are working wonderfully!

![tests passing!](//images.ctfassets.net/hoagspxz8z3s/3i2ffM1orrfeAkFJbNIlE3/30a360b60b08b9b8abbeae3c299d909c/image.png)

### Any other considerations?

This approach works well for small http based functions. There are a few other things you may want to think about for your project:

- What about other events which trigger your cloud functions (PubSub events perhaps?)
- Will your dependencies grow larger overtime? You may need to split functions up to keep dependency sizes down (however, don't run before you can walk.). Dependencies may have a negative impact on your function cold start times.
- How does this impact your deployment strategy? You package the `build/src` directory along with your `package.json` and `yarn.lock`/`package-lock.json`

### Conclusion

There you have it, a simple way to develop your HTTP based cloud functions locally using Typescript! Hopefully this has helped you setup a new project, or reconsider the way you're currently developing cloud functions.

Developing GCP Cloud Functions locally with Typescript

Only a quick post to say I recently passed the GCP Professional Cloud Architect certification! Really happy to have passed, and lines up nicely with plans for the future.

You can view my certification here: https://www.credential.net/c92a1e7a-f9e7-4ba6-93fe-69a7032b472d#gs.hx4hqc

New Certification! GCP Professional Cloud Architect

If you're running PHP-FPM in Kubernetes you have likely found that adding a Horizontal Pod Autoscaler (HPA hereonin) that is based on the CPU ultisation metric isn't reliable. This is because web traffic by nature is somewhat spikey, and scaling based on a sustained high CPU % is in reality too late to scale.

There is a solution however by using Kubernetes Custom Metrics API. Now, this blog post is entirely focused on Google Kubernetes Engine, and we're using Stackdriver as our custom metrics server. I'll explain the setup details in this post. If you're not running GKE there will be a few steps you'll need to swap out.

## First up, how should php-fpm be configured?
The general advice is to ensure the process manager should be configured to `static`, then you're `pm.max_children` value should be relatively low as each pod/container shouldn't run too many processes. You'll need to balance this based on your kubernetes container request/limit values for CPU and Memory.

Additionally we enable the php-fpm status page. This is important, as later on we will scrape the metrics from the status page!

Here's a sample of the configuration I use:

```
pm = static
pm.status_path = /status
pm.max_children = 10
; A child process will handle at least 200 requests before respawning.
; useful for memory leaks in 3rd party code.
pm.max_requests = 200
```

Then the kubernetes deployment configuration:
```
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: phpfpm-demo
  labels:
    app: phpfpm-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: phpfpm-demo
  template:
    metadata:
      labels:
        app: phpfpm-demo
    spec:
      containers:
      - name: fpm
        image:  playsportsgroup/php:7.3-fpm-alpine-root
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9000
        resources:
          requests:
            cpu: 1
            memory: 1G
          limits:
            cpu: 1
            memory: 1G
```

Now, as it stands we have a deployment with a php-fpm container. Next up we'll need to begin scraping our metrics.

## Scraping php-fpm status page metrics

The easiest way to do this is to use a Prometheous compatible exporter. I have personally used this one: [https://github.com/hipages/php-fpm_exporter](https://github.com/hipages/php-fpm_exporter)

We will need to add this container to our Deployment. So let's do that, our new deployment will look like this:

```
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: phpfpm-demo
  labels:
    app: phpfpm-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: phpfpm-demo
  template:
    metadata:
      labels:
        app: phpfpm-demo
    spec:
      containers:
      - name: fpm
        ... (omitted to keep it readible!)
      - name: phpfpm-exporter
        image: hipages/php-fpm_exporter
        ports:
          - name: exporter
            containerPort: 9253
        env:
          - name: PHP_FPM_SCRAPE_URI
            value: "tcp://localhost:9000/status"
          - name: PHP_FPM_FIX_PROCESS_COUNT
            value: "true"
        resources:
          requests:
            cpu: 10m
          limits:
            cpu: 10m
```

This works by scraping the status page over a TCP connection to the php-fpm container. The metrics will be exposed on the port 9253.

Now this isn't it. These metrics are currently not being sent anyway. The exporter has simply made them available! Next up we'll need to convert the Prometheous metrics into Stackdriver Metrics. This can be achieved with another container provided by Google.

```
... (continuing from the pevious)
      - name: prometheus-to-sd
        image: gcr.io/google-containers/prometheus-to-sd:v0.9.2
        ports:
          - name: profiler
            containerPort: 6060
        command:
          - /monitor
          - --stackdriver-prefix=custom.googleapis.com
          - --monitored-resource-type-prefix=k8s_
          - --source=:http://localhost:9253
          - --pod-id=$(POD_NAME)
          - --namespace-id=$(POD_NAMESPACE)
          - --cluster-location=[ADD_YOUR_CLUSTER_LOCATION_HERE]
        resources:
          requests:
            cpu: 10m
          limits:
            cpu: 10m
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
```

In the above example you'll need to define the cluster location yourself. I have found that within a Deployment this value cannot be automatically set. This is just the region that your cluster is in, so `europe-west1` if you're using the Belgium region for example.

Let's take a look at the command arguments we are using here, as this deviates from GCP's documentation here slightly, again this is because we are using a Deployment here. GCP's guides only work when creating Pods directly.

`--stackdriver-prefix=custom.googleapis.com` is required to ensure the metric is tracked in stackdriver as a custom metric. Otherwise we won't be able to scale based on this!

`--monitored-resource-type-prefix=k8s_` by default this appears to set the legacy prefix which is `gke_`. We don't want this as it doesn't work with Deployments!

`--source=:http://localhost:9253` this specifies where the source of our metrics. The colon is defining the namespace, which in this case is empty. This is important because GKE doesn't support scaling on custom metrics that contain slashes, by adding a namespace it will add a slash. i.e. `--source=mynamespace:http://localhost:9253` would create: `custom.googleapis.com/mynamespace/some_metric`, which is not desired!

`--pod-id=$(POD_NAME)` This applies the pod-id label in Stackdriver, allowing us to see the origin of metrics in relation to the pod.

`--namespace-id=$(POD_NAMESPACE)` Same as above, but for the kubernetes namespace your pod belongs to (note in my example deployment I have omitted the namespace which means it will default to `default`).

The last two arguments are using the environment variables defined on the container, these are coming from Kubernetes downward API. The Pod ID will be generated uniquely depending on the revision and number of replicas in your deployment, so we do not know this until runtime.

Ok. With that done we should now see metrics appearing in Stackdriver! Using the Metrics Exlorer you will be able to search for the metrics, they will all be prefixed with `phpfpm_` - a full list can be found here: [https://github.com/hipages/php-fpm_exporter](https://github.com/hipages/php-fpm_exporter)

![stackdriver-metrics-explorer](//images.ctfassets.net/hoagspxz8z3s/1Uz150890U1KF4yVXCi8Za/550336de729125f637f31493e5150feb/image.png)

## Configure cluster to access stackdriver metrics

Now, we have our metrics going to Stackdriver. But we won't be able to access these metrics inside Kubernetes to trigger our autoscaler just yet. We'll need to create the required authorisation roles to do this.

```
kubectl create clusterrolebinding cluster-admin-binding \
    --clusterrole cluster-admin --user "$(gcloud config get-value account)"
```

> Note: the above command will use your user account. I'd recommend using a service account instead that has the specific resources applied. Tying resources users is never a good approach.


With the authorisation role in place we can now deploy the Custom Metrics Stackdriver Adaptor to our cluster. This will make the stackdriver metrics available within your cluster.

```
kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/k8s-stackdriver/master/custom-metrics-stackdriver-adapter/deploy/production/adapter_new_resource_model.yaml
```

__Note:__ This deviates slightly from some of GCP's documentation again. They may tell you to apply a different file from the same github repo. That will be the legacy version which again, does not work with Deployments due to deployments using the `k8s_` prefix for resources instead of the legacy `gke_` (I lost a lot of hours to this...).

## Create your autoscaler

Finally we're now ready to create our autoscaler. For this I'm going to use the `phpfpm_active_processes` metric. As we're using a statically defined number of processes we can trigger scaling when the number of active processes exceed a given average target value.

```
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: phpfpm-demo-autoscaler
spec:
  scaleTargetRef:
    apiVersion: apps/v1beta1
    kind: Deployment
    name: phpfpm-demo
  minReplicas: 1
  maxReplicas: 8
  metrics:
    - type: Pods
      pods:
        metricName: phpfpm_active_processes
        targetAverageValue: 6
```

We have now defined an autoscaler with a minimum of 1 replica, which will increase to a max of 8. This will be triggered when the average value exceeds 6 active processes within phpfpm. Magical.

### Useful links:

- [https://cloud.google.com/kubernetes-engine/docs/tutorials/custom-metrics-autoscaling](https://cloud.google.com/kubernetes-engine/docs/tutorials/custom-metrics-autoscaling)
- [https://github.com/GoogleCloudPlatform/k8s-stackdriver](https://github.com/GoogleCloudPlatform/k8s-stackdriver)
- [https://github.com/craigmj/phpfpm_exporter](https://github.com/craigmj/phpfpm_exporter)

👋 Hey, I’m Ash!

A Software Engineer and keen triathlete.

Here's some recent^* posts of mine...

Handling custom error classes in Typescript

Improved mono-repo for Typescript + Cloud Function development

Provisioning GCP Cloud Functions with Terraform

Developing GCP Cloud Functions locally with Typescript

New Certification! GCP Professional Cloud Architect

Scaling PHP-FPM with custom metrics on GKE/kubernetes

👋 Hey, I’m Ash!

A Software Engineer and keen triathlete.

Here's some recent* posts of mine...

Handling custom error classes in Typescript

Improved mono-repo for Typescript + Cloud Function development

Provisioning GCP Cloud Functions with Terraform

Developing GCP Cloud Functions locally with Typescript

New Certification! GCP Professional Cloud Architect

Scaling PHP-FPM with custom metrics on GKE/kubernetes

Here's some recent^* posts of mine...