How to start a microservice

This article described how to start a new microservice, written in Node.js.

• Create a new repository on GitHub under AEGEE namespace

• Create a new folder on your local machiine

• Install Node.js and Git

• Create a new repo: npm init, then type all the required information. This will create package.json file, which is kind of a manifest for the application.

• Take any of the existing repos for reference, for example https://github.com/AEGEE/oms-events (they mostly have the same structure, see below)

• Install all the deps you will need via npm install (see below)

• Tweak dockerfile and docker compose files (see below)

• Connect it all to oms-docker setup (see below)

• ???

• PROFIT

Now into more details.

Files structure

These files/folders are common for all the modules.

• config/ - folder for storing configs

• lib/ - application data, functions etc.

• lib/bugsnag.js - Bugsnag, the error tracking tool

• lib/constants.js - constants used over the project

• lib/errors.js - errors processing

• lib/helpers.js - helpers used over the project

• lib/logger.js - logging library (Bunyan)

• lib/mailer.js - oms-mailer integration

• lib/metrics.js - Prometheus metrics

• lib/middlewares.js - common middlewares (authorizing, errors processing etc.)

• lib/morgan.js - Morgan (library for logging requests)

• lib/run.js - file to run the whole app

• lib/server.js - list of all endpoints and the place where they are mounted

• lib/sequelize.js - Sequelize, the ORM for Postgres

• docker/ - Docker stuff, see below

• models/ - ORM models for Sequelize

• migrations/ - db migrations

• test/ - everything related to testing

• test/api - testing API requests

• test/assets - all files/other services' responses used for testing

• test/scripts - helpers/stub data generators/mocks

• package.json - app manifesto basically

How Express works

All microservices use Express as a web framework. The idea is the following: on each endpoint (HTTP verb + path) some middlewares are mounted and executed consequently.

Dependencies you may need

You can install deps via npm install <name>

• express - Express router

• sequelize - Sequelize, the ORM for postgres database

• morgan - logging requests

• bunyan - logger

• express-promise-router - a wrapper to have async middlewares

• multer - processing files uploaded

• request + request-promise-native - requests to other services

• moment - working with dates

Dev dependencies (install via npm install -D <name>)

• jest - testing framework

• eslint - static code analysis

• codecov - integration with CodeCov, code coverage service

• nyc - test runner

• nock - mock all external requests

• commitlint - forces you to follow the conventional commit names

• husky - custom git hooks

• semantic-release - used in CI to automatucally bump versions on commits to master

Docker stuff

Everything that is in docker/ folder relates to how the app is executed (all microservices in our setup are set of docker containers). There's a folder structure as well:

• docker/docker-compose.yml - a .yml file which is a manifesto of docker: which containers to run, which volumes to mount, what env params to pass etc.

• docker/docker-compose.dev.yml - same but for dev setup (for dev setup, both docker-compose.yml and then docker-compose.dev.yml are used)

• docker/*/Dockerfile - Dockerfile (basically a sequence of steps used to build the container)

On development mode, you build the container on a host machine.
On production, when you merge to master, a CI should build the container and push it to Dockerhub, then you pull the ready-to-use container on production server and use it.

Also there's a thing called traefik, which processes all requests going to the system, then mapping them to services based on URLs/hosts/headers/whatever. It's configured via docker-compose.yml labels. For example, everything that has '/services/oms-events/api' as a prefix is sent to oms-events with this prefix stripped.

Wiring it all to oms-docker

You do this:

1. edit the oms-docker/.env file and change this:

• add your service to ENABLED_SERVICES

• add PATH_OMS_<insert-your-service-name> vars pointing to your <service-name>/docker folder, which (the variable) should also be in docker/*.yml folder

• make start, then the app should start

If you want to store some static information to store (like users' avatars etc), you'll probably need another nginx container to serve static to end users, check out oms-events and oms-statutory on using it.

Also it's strongly advised to have a lot of tests on a backend so you'd have 100% code coverage everywhere, it helps a lot.

This describes how to start a backend, for frontend this is a (not complete) list to have it running:

• add some new components

• add some new menu entries

• add it to router to process some URLs on the frontend

• test it together with the backend to validate it's running

• make a PR to oms-frontend, wait till it's merged

• ???

• PROFIT