Creating RESTful services with Go — Part 2

In this blog, we are going to be building a fully-fledged REST API that exposes GET, POST, DELETE, and PUT endpoints that will subsequently allow you to perform the full range of CRUD operations. As mentioned in the previous article , the blog series will encapsulate a production-ready project. Thus, this part of the blog will lay the groundwork for an online coffee shop.

The code for this part of the blog will be hosted here: https://github.com/nandangrover/go-microservices . The second part of the series, which is what this blog documents can be found in this branch .

To keep this simple and focus on the basic concepts, we won’t be interacting with any backend database technologies to store the products that we’ll be playing with. However, we will be writing this REST API in such a way that it will be easy to update the functions we will be defining so that they make subsequent calls to a database to perform any necessary CRUD operations.

What are the RESTful services?

REST is an architectural approach for designing web services. REST APIs are designed around resources, which are any kind of object, data, or service that can be accessed by the client. A resource has an identifier, which is a URI that uniquely identifies that resource. For example, the URI for a particular customer order might be:

[https://adventure-works.com/orders/1](https://adventure-works.com/orders/1)

Clients interact with a service by exchanging representations of resources. Many web APIs use JSON(it’s not required though, of course) as the exchange format. For example, a GET request to the URI listed above might return this response body:

{“orderId”:1,“orderValue”:99.90,“productId”:1,“quantity”:1}

The HTTP protocol defines several methods that assign semantic meaning to a request. The common HTTP methods used by most RESTful web APIs are:

• **GET **retrieves a representation of the resource at the specified URI. The body of the response message contains the details of the requested resource.

• **POST **creates a new resource at the specified URI. The body of the request message provides the details of the new resource. Note that POST can also be used to trigger operations that don’t actually create resources.

• PUT either creates or replaces the resource at the specified URI. The body of the request message specifies the resource to be created or updated.

• PATCH performs a partial update of a resource. The request body specifies the set of changes to apply to the resource.

• DELETE removes the resource at the specified URI.

We could have used GraphQL or a gRPC architecture for building our microservices structure. So why didn’t we? Well REST is comparatively simpler to implement. In future blogs, I will look into redesigning the backend with some of the above mentioned technologies.

What’s the file structure like?

In the last blog, we added two new handlers called hello and goodbye. We don’t need those anymore, so we have deleted them. Instead, we create a new handler called products.go. We will perform our CRUD operations through this handler. Since we are creating a coffee shop, we need a data store that stores our products. **Products.go **will store the fields of the product to be stored as a go struct.

Product Handler and Data Store

Let’s look at our handler first. We start by checking for the HTTP verb that was requested by the API, namely: GET, POST, and PUT. We also have written methods on the Products struct. These can be called by p.MethodName. This helps a lot in abstracting away our logic and in reusability. If you look at the underlying structure of the code, you will see some tenets of functional programming. Golang is not a functional language but has a lot of features that enable us to applies functional principles in the development, turning our code more elegant, concise, maintainable, easier to understand, and test.

package handlers

import (
 "log"
 "net/http"
 "regexp"
 "strconv"

"github.com/nandangrover/go-microservices/data"
)

//Products structure that holds a logger
type Products struct {
 l *log.Logger
}

// NewProducts function return the pointer to Products structure
func NewProducts(l *log.Logger) *Products {
 return &Products{l}
}

func (p *Products) ServeHTTP(rw http.ResponseWriter, r *http.Request) {
 if r.Method == http.MethodGet {
  p.getProducts(rw, r)
  return
 }
 if r.Method == http.MethodPost {
  p.addProduct(rw, r)
  return
 }
 if r.Method == http.MethodPut {
  // expect the id in the URI
  regex := regexp.MustCompile(`/([0-9]+)`)
  group := regex.FindAllStringSubmatch(r.URL.Path, -1)

if len(group) != 1 || len(group[0]) != 2 {
   http.Error(rw, "Invalid URI", http.StatusBadRequest)
   return
  }

idString := group[0][1]
  // Ignore the error for now
  id, _ := strconv.Atoi(idString)

p.updateProducts(id, rw, r)
 }
 // catch all other http verb with 405
 rw.WriteHeader(http.StatusMethodNotAllowed)
}

func (p *Products) getProducts(rw http.ResponseWriter, r *http.Request) {
 p.l.Println("Handle GET products")

listOfProducts := data.GetProducts()
 // Use encoder as it is marginally faster than json.marshal. It's important when we use multiple threads
 // d, err := json.Marshal(listOfProducts)
 err := listOfProducts.ToJSON(rw)
 if err != nil {
  http.Error(rw, "Unable to marshal json", http.StatusInternalServerError)
 }
}

func (p *Products) addProduct(rw http.ResponseWriter, r *http.Request) {
 p.l.Println("Handle POST product")

prod := &data.Product{}
 // The reason why we use a buffer reader is so that we don't have to allocate all the memory instantly to a slice or something like that,
 err := prod.FromJSON(r.Body)
 if err != nil {
  http.Error(rw, "Unable to unmarshal json", http.StatusBadRequest)
 }
 // p.l.Printf("Prod %#v", prod)
 data.AddProduct(prod)
}

func (p *Products) updateProducts(id int, rw http.ResponseWriter, r *http.Request) {
 p.l.Println("Handle Put product")

prod := &data.Product{}
 // The reason why we use a buffer reader is so that we don't have to allocate all the memory instantly to a slice or something like that,
 err := prod.FromJSON(r.Body)
 if err != nil {
  http.Error(rw, "Unable to unmarshal json", http.StatusBadRequest)
 }

err = data.UpdateProduct(id, prod)
 if err == data.ErrProductNotFound {
  http.Error(rw, "Product not found", http.StatusNotFound)
  return
 }

if err != nil {
  http.Error(rw, "Product not found", http.StatusInternalServerError)
  return
 }

}

**Data Store **for our products defines the structure that each of the coffee shop products will have. We need the product to be exported so each of the keys inside the Product struct needs to have an uppercase first character.

We also store some helper utility methods in this file, such as ToJSON and FromJSON. These methods help convert our Product Struct to JSON and vice versa. Abstraction is definitely possible here, but we will look at it more in the next blog.

Finally, we also have our variable productList which stores a slice of the reference to Product struct. Inside the slice, we have added some dummy data which can be used to perform our CRUD operations.

package data

import (
 "encoding/json"
 "fmt"
 "io"
 "time"
)

//Product defines the structure for an API product
//Since encoding/json is a package residing outside our package we need to uppercase the first character of the fields inside the structure
//To get nice json field names we can add struct tags though. This will output the key name as the tag name
type Product struct {
 ID          int     `json:"id"`
 Name        string  `json:"name"`
 Description string  `json:"description"`
 Price       float32 `json:"price"`
 SKU         string  `json:"sku"`
 CreatedOn   string  `json:"-"`
 UpdatedOn   string  `json:"-"`
 DeletedOn   string  `json:"-"`
}

// Products is a type defining slice of struct Product
type Products []*Product

// ToJSON is a Method on type Products (slice of Product), used to covert structure to JSON
func (p *Products) ToJSON(w io.Writer) error {
 // NewEncoder requires an io.Reader. http.ResponseWriter is the same thing
 encoder := json.NewEncoder(w)
 return encoder.Encode(p)
}

// FromJSON is a Method on type Products (slice of Product)
func (p *Product) FromJSON(r io.Reader) error {
 decoder := json.NewDecoder(r)
 return decoder.Decode(p)
}

//GetProducts - Return the product list
func GetProducts() Products {
 return productList
}

//AddProduct - Add the product to our struct Product
func AddProduct(p *Product) {
 p.ID = getNextID()
 productList = append(productList, p)
}

//UpdateProduct - Updates the product to our struct Product
func UpdateProduct(id int, p *Product) error {
 _, pos, err := findProduct(id)
 if err != nil {
  return err
 }

p.ID = id
 productList[pos] = p

return nil
}

func findProduct(id int) (*Product, int, error) {
 for i, p := range productList {
  if p.ID == id {
   return p, i, nil
  }
 }
 return nil, -1, ErrProductNotFound
}

// ErrProductNotFound is the Standard Product not found error structure
var ErrProductNotFound = fmt.Errorf("Product not found")

// Increments the Product ID by one
func getNextID() int {
 lastProduct := productList[len(productList)-1]
 return lastProduct.ID + 1
}

var productList = []*Product{
 &Product{
  ID:          1,
  Description: "Latte",
  Name:        "Milky coffee",
  SKU:         "abc323",
  Price:       200,
  UpdatedOn:   time.Now().UTC().String(),
  CreatedOn:   time.Now().UTC().String(),
 },
 &Product{
  ID:          2,
  Description: "Expresso",
  Name:        "Strong coffee",
  SKU:         "errfer",
  Price:       150,
  UpdatedOn:   time.Now().UTC().String(),
  CreatedOn:   time.Now().UTC().String(),
 },
}

Retrieving a Product — GET

To retrieve our product we can send a request through our Unix based terminal in this way:

curl -v localhost:9090 | jq

The jq helps in formatting the response.

The products.go handler is activated upon this request. Inside the ServeHTTP method, we have written an if condition which checks for the HTTP verb that was requested with the http.MethodGet, which is essentially the string “GET”.

if r.Method == http.MethodGet {
  p.getProducts(rw, r)
  return
 }

The getproducts() method is called in turn for the GET request. We send our HTTP ResponseWriter and Request to this method. This method in turn fetches the productList slice from our data store. Since it’s a slice, we convert it to JSON using our ToJSON utility method defined on the type Products which is a slice of struct Product.

func (p *Products) getProducts(rw http.ResponseWriter, r *http.Request) {
 p.l.Println("Handle GET products")

listOfProducts := data.GetProducts()
 // Use encoder as it is marginally faster than json.marshal. It's important when we use multiple threads
 // d, err := json.Marshal(listOfProducts)
 err := listOfProducts.ToJSON(rw)
 if err != nil {
  http.Error(rw, "Unable to marshal json", http.StatusInternalServerError)
 }
}

Encode writes the JSON encoding of v to the stream, followed by a newline character. We could have used json.Marshal here, but we didn’t. Encoder and decoder write struct to slice of a stream or read data from a slice of a stream and convert it into a struct. Internally, it also implements the marshal method. The only difference is if you want to play with string or bytes use marshal, and if any data you want to read or write to some writer interface(such as our ResoponseWriter), use encodes and decode. This in turn is also faster. We won’t notice any speed difference with a single API call, but thousands of simultaneous API calls will be handled better with an encoder instead of the Marshal. You can read more about encoding/json package here .

// ToJSON is a Method on type Products (slice of Product), used to covert structure to JSON
func (p *Products) ToJSON(w io.Writer) error {
 // NewEncoder requires an io.Reader. http.ResponseWriter is the same thing
 encoder := json.NewEncoder(w)
 return encoder.Encode(p)
}

Thus, this encoder writes our response to the ResponseWriter. The final output for our API call is somewhat like this:

[
  {
    "id": 1,
    "name": "Milky coffee",
    "description": "Latte",
    "price": 200,
    "sku": "abc323"
  },
  {
    "id": 2,
    "name": "Strong coffee",
    "description": "Expresso",
    "price": 150,
    "sku": "errfer"
  }
]

Adding a new Product — POST

To add a new product we can send a request through our Unix based terminal in this way:

curl -v localhost:9090 -XPOST -d {"name": "Tea", "description": "Cuppa Tea", "price": 10}

The structure for adding a product is similar to fetching a product. We follow the same flow of data transfer: handler to the data store. The handler recognizes the HTTP verb and calls the appropriate method, i.e, addProduct.

This method in turn creates a reference to our Product struct which defines the structure for our Products. The JSON data received in the request body is sent to the utility method FromJSON to decode it into a structured reference to our defined struct, i.e, Product.

func (p *Products) addProduct(rw http.ResponseWriter, r *http.Request) {
 p.l.Println("Handle POST product")

prod := &data.Product{}
 // The reason why we use a buffer reader is so that we don't have to allocate all the memory instantly to a slice or something like that,
 err := prod.FromJSON(r.Body)
 if err != nil {
  http.Error(rw, "Unable to unmarshal json", http.StatusBadRequest)
 }
 // p.l.Printf("Prod %#v", prod)
 data.AddProduct(prod)
}

Now that we have the product stored as a reference to the struct Product, we append it to our productList slice, which is working as our temporary database storage.

func AddProduct(p *Product) {
 p.ID = getNextID()
 productList = append(productList, p)
}

A new id is generated for our product and it is appended to our slice. If we send a GET request to our product API we will see 3 products listed now, instead of the 2.

[
  {
    "id": 1,
    "name": "Milky coffee",
    "description": "Latte",
    "price": 200,
    "sku": "abc323"
  },
  {
    "id": 2,
    "name": "Strong coffee",
    "description": "Expresso",
    "price": 150,
    "sku": "errfer"
  },

{
    "id": 3,
    "name": "Tea",
    "description": "Cuppa Tea",
    "price": 10
  }
]

Updating an existing Product — PUT

To update an existing product we can send a request through our Unix based terminal in this way:

curl -v localhost:9090/2 -XPUT -d {"name": "Frappuccino", "description": "Cuppa frappuccino", "price": 100}

The structure for updating a product is similar to fetching a product. We follow the same flow of data transfer: handler to the data store. The handler recognizes the HTTP verb and calls the appropriate method, i.e, updateProducts.

PUT requests are a little harder to parse than a simple POST or GET, as we have to extract the requested ID from the URI. How do we do that? We use some regexp.

Since our ID is a number we write a regexp to search for a group of numbers (0–9) which can be repeated, signified by the + token. On running the FindAllStringSubmatch, if we get a successful match, the matched string resides inside a multidimensional array. We extract the necessary group from the index [0][1]. I will leave it up to you to figure out why the index resides in [1] instead of [0].

Regexp is really interesting and can be helpful in a lot of ways. You can read more about the methods Golang’s standard library offer, here .

if r.Method == http.MethodPut {
  // expect the id in the URI
  regex := regexp.MustCompile(`/([0-9]+)`)
  group := regex.FindAllStringSubmatch(r.URL.Path, -1)

if len(group) != 1 || len(group[0]) != 2 {
   http.Error(rw, "Invalid URI", http.StatusBadRequest)
   return
  }

idString := group[0][1]
  // Ignore the error for now
  id, _ := strconv.Atoi(idString)

p.updateProducts(id, rw, r)
 }

Now that we have the product id that we need to update, we can extract the updated information from our request body.

We will decode the product information from our body into a struct reference pointed to the Product inside the data store. This step is similar to the POST request. The **UpdateProduct **method inside our data store is called next.

func (p *Products) updateProducts(id int, rw http.ResponseWriter, r *http.Request) {
 p.l.Println("Handle Put product")

prod := &data.Product{}
 // The reason why we use a buffer reader is so that we don't have to allocate all the memory instantly to a slice or something like that,
 err := prod.FromJSON(r.Body)
 if err != nil {
  http.Error(rw, "Unable to unmarshal json", http.StatusBadRequest)
 }

err = data.UpdateProduct(id, prod)
 if err == data.ErrProductNotFound {
  http.Error(rw, "Product not found", http.StatusNotFound)
  return
 }

if err != nil {
  http.Error(rw, "Product not found", http.StatusInternalServerError)
  return
 }

}

To update our product we need to find it first. It’s a good thing we have the ID from the URI, as that’s the only unique identifier for our Product. We do this by iterating over our **productList **slice and returning the product reference, it’s Index in the slice, and an error (nil if the product is found). We use this index (pos is the variable name for this index), to replace the reference with the JSON received in our request body (obviously it’s decoded as a reference to the product struct now).

//UpdateProduct - Updates the product to our struct Product
func UpdateProduct(id int, p *Product) error {
 _, pos, err := findProduct(id)
 if err != nil {
  return err
 }

p.ID = id
 productList[pos] = p

return nil
}

func findProduct(id int) (*Product, int, error) {
 for i, p := range productList {
  if p.ID == id {
   return p, i, nil
  }
 }
 return nil, -1, ErrProductNotFound
}

The product at index 2 should have been updated now. If we send a GET request to our product API we will see the second product updated with the new values.

[
  {
    "id": 1,
    "name": "Milky coffee",
    "description": "Latte",
    "price": 200,
    "sku": "abc323"
  },
  {
    "id": 2,
    "name": "Frappuccino",
    "description": "Cuppa Frappuccino",
    "price": 100,
    "sku": "errfer"
  },

{
    "id": 3,
    "name": "Tea",
    "description": "Cuppa Tea",
    "price": 10
  }
]

What’s next?

We will be looking into the infamous gorilla framework and maybe some cool documenting techniques with swagger. We will also look into handling files in the next blog. Unit tests and full database integration are obviously on the horizon but we will first look into making a strong foundation to build upon.

References

• Codebase: https://github.com/nandangrover/go-microservices

• Part 2 GitHub link:https://github.com/nandangrover/go-microservices/tree/restful_services_2

• Freecodecamp tutorial for Golang: https://www.youtube.com/watch?v=YS4e4q9oBaU&ab_channel=freeCodeCamp.org

• Rest API design principles: https://docs.microsoft.com/en-us/azure/architecture/best-practices/api-design

• JSON package Golang: https://golang.org/pkg/encoding/json/

• REGEXP package Golang: https://golang.org/pkg/regexp/ Creating Microservices With Go- Part 1 *Coming from a JavaScript background, I have always wanted to learn a static-typed programming language, earlier this…*medium.com