My shot at RESTful Microservices in Rust - Part 3

Part 3 - Linking REST endpoint and db layer

Welcome to part 3 of my Rust microservices series! If you haven't read parts 1 or 2, here are the respective links: part 1 part 2. In this installment I'm going to connect the REST endpoint with the database layer and take care of serialization and deserialization of the Rust structs.

JSON serialization

There are several crates that give you automatic serialization and deserialization of structs to JSON strings. I'm going to use Serde in this PoC. Serde is divided into a core crate and one additional crate per source/target format. So I'm going to use the crates serde, serde_derive and serde_json. The crate serde_derive contains the Serialize and Deserialize macros that implement the trais with same names. This enables us to serialize a struct by calling serde_json::to_string.

src/models/game.rs:

#[derive(Debug, Serialize)]
pub struct DbGame { /* omitted. */ }
 
#[derive(Debug, Serialize)]
pub struct Dimensions { /* omitted. */ }

src/main.rs:

#[macro_use] extern crate serde_derive;
extern crate serde_json;
 
fn main() {
    for game in dao::get_games() {
        println!("{:?}", serde_json::to_string(&game).unwrap());
    }
}

Unsurprisingly, deserialization works the same way.

Connecting the REST endpoint to the database

I'm gonna create a simple endpoint listening on GET /games that will return a list of all games. src/main.rs:

fn main() {
    let mut server = Nickel::new();
    server.get("/games", middleware! {|_req, mut resp|
        resp.set(MediaType::Json);
        let games = dao::get_games();
        serde_json::to_string(&games).unwrap()
    });
    server.listen("0.0.0.0:8080")
        .expect("Error starting server");
}

When I cURL this endpoint I get

< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Sun, 05 Mar 2017 18:26:36 GMT
< Server: Nickel
< Transfer-Encoding: chunked
<
* Connection #0 to host localhost left intact
[{"id":1,"dimensions":{"x":3,"y":3}},{"id":2,"dimensions":{"x":4,"y":5}}]

Deserializing JSON

So now that we've got a working endpoint that lists all the games, let's add one that actually creates a game. I'm gonna keep things simple here and let the caller choose the id of the game and not care about key uniqueness issues for this PoC. The first step is to add the Deserialize macro to the entity structs. After that it's mostly about the dao and the controller code.

src/main.rs:

// ...
server.post("/games", middleware! {|req, mut resp|
    match get_game_from_request(req) {
        Ok(game) => {
            resp.set(StatusCode::Created);
            dao::create_game(game);
            "Ok!".to_string()
        },
        Err(e) => {
            resp.set(StatusCode::BadRequest);
            e
        }
    }
});
// ...
fn get_game_from_request(
    req: &mut nickel::Request,
) -> Result<DbGame, String> {
    let mut body = String::new();
    req.origin.read_to_string(&mut body).unwrap();
    serde_json::from_str::<DbGame>(&body)
        .map_err(|e| e.description().to_string() )
}

Nickel provides built-in JSON deserialization, but this feature relies on the rustc_serialize crate, which I'm not using. Serde is a newer and more modular implementation for serialization and deserialization. The get_game_from_request function extracts the body from the request and then tries to deserialize it. The database access code is straight-forward:

game_dao.rs:

pub fn create_game(game: DbGame) {
    let conn = connect();
    conn.execute(r#"
        INSERT INTO games (id, dimension_x, dimension_y)
        VALUES ($1, $2, $3)"#,
        &[&game.id, &game.dimensions.x, &game.dimensions.y]
    ).expect("Error inserting into database");
}

As promised at the beginning, I don't care about primary key uniqueness in this PoC, so if you try to POST a game with an id that's already there, the thread is going to panic.

Conclusions

We've seen that it is possible to create microservices in Rust with little effort, even though compared to older languages there's more boilerplate code that you have to write yourself. Especially Nickel seems to have a lot of room for improvement. I don't like that you seem to have to return a String from every endpoint definition in the middleware! macro, but then I'm not very good at reading macro definitions in Rust yet.

One could think that interacting with postgres directly and not using an OR-Mapper is a bad idea, but I think that especially in microservices, the number of entities is usually small enough for that not to matter too much.

This concludes the third and last part of this proof of concept. You can find the source code here. Thanks for reading and please feel free to comment.

My shot at RESTful Microservices in Rust - Part 2

Part 2 - Database interaction

Welcome back! If you haven't read part 1 yet: this series of blog posts is about creating a simple RESTful service in Rust. After setting up the project in part 1, I'm gonna set up a basic database interaction, to make the scenario more realistic.

I initially wanted to use a full-fledged ORM solution for this PoC but then decided it's better to concentrate on a few things at a time. To put it in a nutshell, for this project I use Diesel's migration features without the actual OR-mapping.

Diesel setup

Diesel comes as a library and additionally as a tool for the command line, called diesel_cli. I install the command line tool with cargo install diesel_cli.

For diesel to know how to connect to the database I add a .env file to the project:

DATABASE_URL=postgres://postgres@localhost/battleship

The .env file is just a means of collecting environment variables and it can easily incorporated into your program with the dotenv library.

Now i need to create a database. I chose to just spin up a dockerized postgres server for development purposes like so:

docker run \
  -d --name battleship_db \
  -p 5432:5432 -e POSTGRES_PASSWORD='' \
  postgres

When I now run diesel setup two things happen:

1. a migrations directory is created
2. the battleship database is created inside the postgres container

A database migration

Now that there is a database, I'll create a migration to initialize the database with a table. I run diesel migration generate create_games, which creates two files in migrations/20170301195954_create_games/: up.sql and down.sql. Unsurprisingly, one of them is used to make a change in the database, whereas the other reverts the change.

up.sql:

CREATE TABLE games (
  id BIGSERIAL NOT NULL PRIMARY KEY,
  dimension_x INTEGER NOT NULL,
  dimension_y INTEGER NOT NULL
);

down.sql:

DROP TABLE games;

I now run diesel migration run and up.sql is executed in the dockerized database.

The model

I need a representation of a game in Rust, so I create the following structs:

src/models/game.rs:

#[derive(Debug)]
pub struct DbGame {
    pub id: i64,
    pub dimensions: Dimensions,
}
 
#[derive(Debug)]
pub struct Dimensions {
    pub x: i32,
    pub y: i32,
}

Interacting with the db

Since I'm not using an OR-Mapper, I'm gonna query the database through plain SQL, using the native postgres driver (added to Cargo.toml). Futhermore, I'll use dotenv to get the database connection URL from .env.

I create a method that establishes a database connection and another one that queries the games table for all entries. The latter iterates over the results and maps each row to a DbGame using one of the standard type conversion mechanisms in Rust, the From trait. For this to work, there must be an implementation of From<Row> for DbGame, which is listed below.

src/dao/game_dao.rs:

use dotenv::dotenv;
use models::DbGame;
use postgres::{Connection, TlsMode};
use std::env;
 
fn connect() -> Connection {
    dotenv().ok();
    let database_url = env::var("DATABASE_URL")
        .expect("DATABASE_URL must be set");
    Connection::connect(&*database_url, TlsMode::None)
        .expect(&format!("Error connecting to {}", &database_url))
}
 
pub fn get_games() -> Vec<DbGame> {
    let conn = connect();
    let rows = conn.query("SELECT * FROM games", &[])
        .expect("Error querying database");
 
    rows.iter()
        .map(DbGame::from)
        .collect()
}

src/models/game.rs:

impl<'a> From<Row<'a>> for DbGame {
    fn from(row: Row) -> Self {
        DbGame {
            id: row.get("id"),
            dimensions: Dimensions {
                x: row.get("dimension_x"),
                y: row.get("dimension_y"),
            },
        }
    }
}

I can then list the database entries in main.rs:

fn main() {
    for game in dao::get_games() {
        println!("{:?}", game);
    }
}

Which yields the following output for me, after I've manually inserted some data:

DbGame { id: 1, dimensions: Dimensions { x: 3, y: 3 } }
DbGame { id: 2, dimensions: Dimensions { x: 4, y: 5 } }

This concludes part 2 of the PoC. In part 3 I will show how I connected the database layer with the REST endpoint and how to convert the Rust structs into JSON.

My shot at RESTful Microservices in Rust - Part 1

Part 1 - Getting started

So I'm starting my own tech blog, and the first topic I'd like to cover is Rust, or more specifically, how I go about building a RESTful microservice in Rust. Bear in mind that this is not a tutorial, it's me telling the story of how I went about it what I think about the result.

I'll name this PoC project rest-battleship, because a lot of my experiments with the Rust language have been about this classic game, for reasons. The complete code can be found on GitHub.

What's to be done?

The minimum requirements I have for this PoC are

• A JSON API
• REST- and meaningful responses, i.e. using appropriate HTTP response codes
• Database interaction

Part 1 will cover setting up the project and getting a minimal HTTP service up and running.

Setting up the project

For this project I'll be using Rust 1.15.1, being the latest stable release at the time of writing. If you haven't got Rust installed yet, it's a breeze with rustup. Version 1.15 has been a kind of milestone for Rust, as a long-awaited feature as become stable: custom derive. Custom derive allows you to create macros that can be used in Rust's #[derive()] attribute. This means you can finally generate custom code for structs on the stable branch of the language.

So cargo new --bin rest-battleship creates the project with a Cargo.toml file to describe the thing and a src/main.rs that will print 'Hello, world!' - easy!

A minimal HTTP service

Okay, so the next step is to find a framework that let's us serve HTTP requests. There are several ones available and I've chosen nickel for this PoC. So let's add this dependency to Cargo.toml:

[package]
name = "rest-battleship"
version = "0.1.0"
authors = ["René Perschon <notmyemail@gmail.com>"]
 
[dependencies]
nickel = "0.9.0"

And then change src/main.rs to start a server and listen on port 8080:

#[macro_use] extern crate nickel;
 
use nickel::Nickel;
use nickel::HttpRouter;
 
fn main() {
    let mut server = Nickel::new();
    server.get("/games", middleware! {|_req, _resp|
        "Hello, world!"
    });
    server.listen("0.0.0.0:8080").expect("Error starting server");
}

Start the service with cargo run and that's enough to cURL http://localhost:8080/games and receive a greeting to the whole world.

This concludes part 1 of the PoC. In part 2 I will cover basic database interaction.