Building a Go Microservice with CI/CD

The TestMain function sets Gin to use the test mode and calls the rest of the test functions. The getRouter function creates and returns a router in a manner similar to the main application. The saveLists() function saves the original article list in a temporary variable. This temporary variable is used by the restoreLists() function to restore the article list to its initial state after a unit test is executed.

Finally, the testHTTPResponse function executes the function passed in to see if it returns a boolean true value - indicating a successful test, or not. This function helps us avoid duplicating the code needed to test the response of an HTTP request.

To check the HTTP code and the returned HTML, we’ll do the following:

1. Create a new router,
2. Define a route to use the same handler that the main app uses (showIndexPage),
3. Create a new request to access this route,
4. Create a function that processes the response to test the HTTP code and HTML, and
5. Call testHTTPResponse() with this new function to complete the test.

Creating the Route Handler

1. Fetches the list of articles

We will create all route handlers for article related functionality in the handlers.article.go file. The handler for the index page, showIndexPage performs the following tasks:

This can be done using the getAllArticles function defined previously:

2. Renders the index.html template passing it the article list

articles := getAllArticles()

This can be done using the code below:

c.HTML( // Set the HTTP status to 200 (OK) http.StatusOK, // Use the index.html template "index.html", // Pass the data that the page uses gin.H{ "title": "Home Page", "payload": articles, }, )

The only difference from the version in the previous section is that we’re passing the list of articles which will be accessed in the template by the variable named payload.

The handlers.article.go file should contain the following code:

// handlers.article.go package main import ( "net/http" "github.com/gin-gonic/gin" ) func showIndexPage(c *gin.Context) { articles := getAllArticles() // Call the HTML method of the Context to render a template c.HTML( // Set the HTTP status to 200 (OK) http.StatusOK, // Use the index.html template "index.html", // Pass the data that the page uses gin.H{ "title": "Home Page", "payload": articles, }, ) }

If you now build and run your application and visit http://localhost:8080 in a browser, it should look like this:

These are the new files added in this section:

├── common_test.go ├── handlers.article.go ├── handlers.article_test.go ├── models.article.go └── models.article_test.go

Displaying a Single Article

In the last section, while we displayed a list of articles, the links to the articles didn’t work. In this section, we’ll add handlers and templates to display an article when it is selected.

Setting Up the Route

We can set up a new route to handle requests for a single article in the same manner as in the previous route. However, we need to account for the fact that while the handler for all articles would be the same, the URL for each article would be different. Gin allows us to handle such conditions by defining route parameters as follows:

router.GET("/article/view/:article_id", getArticle)

This route will match all requests matching the above path and will store the value of the last part of the route in the route parameter named article_id which we can access in the route handler. For this route, we will define the handler in a function named getArticle.

The updated main.go file should contain the following code:

func main() { router := gin.Default() router.LoadHTMLGlob("templates/*") // Handle Index router.GET("/", showIndexPage) // Handle GET requests at /article/view/some_article_id router.GET("/article/view/:article_id", getArticle) router.Run() } . . .

Creating the View Templates

We need to create a new template at templates/article.html to display the content of a single article. This can be created in a manner similar to the index.html template. However, instead of the payload variable containing the list of articles, in this case it will contain a single article.

Create the article template in templates/article.html:

<!--article.html--> <!--Embed the header.html template at this location--> {{ template "header.html" .}} <!--Display the title of the article--> <h1>{{.payload.Title}}</h1> <!--Display the content of the article--> <p>{{.payload.Content}}</p> <!--Embed the footer.html template at this location--> {{ template "footer.html" .}}

Specifying the Requirement for the Go Microservice Router

The test for the handler of this route will check for the following conditions:

• The handler responds with an HTTP status code of 200,
• The returned HTML contains a title tag containing the title of the article that was fetched.

The code for the test will be placed in the TestArticleUnauthenticated function in the handlers.article_test.go file. We will place helper functions used by this function in the common_test.go file.

Creating the Route Handler

1. Extracts the ID of the article to display

The handler for the article page, getArticle performs the following tasks:

To fetch and display the right article, we first need to extract its ID from the context. This can be extracted as follows:

c.Param("article_id")

2. Fetches the article

where c is the Gin Context which is a parameter to any route handler when using Gin.

This can be done using the getArticleByID() function defined in the models.article.go file:

article, err := getArticleByID(articleID)

After adding getArticleByID, the models.article.go file should look like this:

// models.article.go package main import ( "errors" ) type article struct { ID int `json:"id"` Title string `json:"title"` Content string `json:"content"` } // For this demo, we're storing the article list in memory // In a real application, this list will most likely be fetched // from a database or from static files var articleList = []article{ article{ID: 1, Title: "Article 1", Content: "Article 1 body"}, article{ID: 2, Title: "Article 2", Content: "Article 2 body"}, } // Return a list of all the articles func getAllArticles() []article { return articleList } func getArticleByID(id int) (*article, error) { for _, a := range articleList { if a.ID == id { return &a, nil } } return nil, errors.New("Article not found") }

3. Renders the article.html template passing it the article

This function loops through the article list and returns the article whose ID matches the ID passed in. If no matching article is found it returns an error indicating the same.

This can be done using the code below:

c.HTML( // Set the HTTP status to 200 (OK) http.StatusOK, // Use the article.html template "article.html", // Pass the data that the page uses gin.H{ "title": article.Title, "payload": article, }, )

The updated handlers.article.go file should contain the following code:

// handlers.article.go package main import ( "net/http" "strconv" "github.com/gin-gonic/gin" ) func showIndexPage(c *gin.Context) { articles := getAllArticles() // Call the HTML method of the Context to render a template c.HTML( // Set the HTTP status to 200 (OK) http.StatusOK, // Use the index.html template "index.html", // Pass the data that the page uses gin.H{ "title": "Home Page", "payload": articles, }, ) } func getArticle(c *gin.Context) { // Check if the article ID is valid if articleID, err := strconv.Atoi(c.Param("article_id")); err == nil { // Check if the article exists if article, err := getArticleByID(articleID); err == nil { // Call the HTML method of the Context to render a template c.HTML( // Set the HTTP status to 200 (OK) http.StatusOK, // Use the index.html template "article.html", // Pass the data that the page uses gin.H{ "title": article.Title, "payload": article, }, ) } else { // If the article is not found, abort with an error c.AbortWithError(http.StatusNotFound, err) } } else { // If an invalid article ID is specified in the URL, abort with an error c.AbortWithStatus(http.StatusNotFound) } }

If you now build and run your application and visit http://localhost:8080/article/view/1 in a browser, it should look like this:

The new files added in this section are as follows:

└── templates └── article.html

Responding With JSON/XML

In this section, we will refactor the application a bit so that, depending on the request headers, our application can respond in HTML, JSON or XML format.

Creating a Reusable Function

So far, we’ve been using the HTML method of Gin's context to render directly from route handlers. This is fine when we always want to render HTML. However, if we want to change the format of the response based on the request, we should refactor this part out into a single function that takes care of the rendering. By doing this, we can let the route handler focus on validation and data fetching.

A route handler has to do the same kind of validation, data fetching and data processing irrespective of the desired response format. Once this part is done, this data can be used to generate a response in the desired format. If we need an HTML response, we can pass this data to the HTML template and generate the page. If we need a JSON response, we can convert this data to JSON and send it back. Likewise for XML.

We’ll create a render function in main.go that will be used by all the route handlers. This function will take care of rendering in the right format based on the request's Accept header.

In Gin, the Context passed to a route handler contains a field named Request. This field contains the Header field which contains all the request headers. We can use the Get method on Header to extract the Accept header as follows:

// c is the Gin Context c.Request.Header.Get("Accept")

• If this is set to application/json, the function will render JSON,
• If this is set to application/xml, the function will render XML, and
• If this is set to anything else or is empty, the function will render HTML.

The render function is as follows, add it in the handlers.article.go file:

// Render one of HTML, JSON or CSV based on the 'Accept' header of the request // If the header doesn't specify this, HTML is rendered, provided that // the template name is present func render(c *gin.Context, data gin.H, templateName string) { switch c.Request.Header.Get("Accept") { case "application/json": // Respond with JSON c.JSON(http.StatusOK, data["payload"]) case "application/xml": // Respond with XML c.XML(http.StatusOK, data["payload"]) default: // Respond with HTML c.HTML(http.StatusOK, templateName, data) } }

Modifying the Requirement for the Route Handlers With a Unit Test

Since we are now expecting JSON and XML responses if the respective headers are set, we should add tests to the handlers.article_test.go file to test these conditions. We will add tests to:

1. Test that the application returns a JSON list of articles when the Accept header is set to application/json
2. Test the application returns an article in XML format when the Accept header is set to application/xml

These will be added as functions named TestArticleListJSON and TestArticleXML.

Updating the Route Handlers

The route handlers don’t really need to change much as the logic for rendering in any format is pretty much the same. All that needs to be done is use the render function instead of rendering using the c.HTML methods.

For example, the showIndexPage route handler in handler.article.go will change from:

func showIndexPage(c *gin.Context) { articles := getAllArticles() // Call the HTML method of the Context to render a template c.HTML( // Set the HTTP status to 200 (OK) http.StatusOK, // Use the index.html template "index.html", // Pass the data that the page uses gin.H{ "title": "Home Page", "payload": articles, }, ) }

to:

Retrieving the List of Articles in JSON Format

func showIndexPage(c *gin.Context) { articles := getAllArticles() // Call the render function with the name of the template to render render(c, gin.H{ "title": "Home Page", "payload": articles}, "index.html") }

To see our latest updates in action, build and run your application. Then execute the following command:

curl -X GET -H "Accept: application/json" http://localhost:8080/

This should return a response as follows:

[{"id":1,"title":"Article 1","content":"Article 1 body"},{"id":2,"title":"Article 2","content":"Article 2 body"}]

Retrieving an Article in XML Format

As you can see, our request got a response in the JSON format because we set the Accept header to application/json.

Let’s now get our application to respond with the details of a particular article in the XML format. To do this, first, start your application as mentioned above. Now execute the following command:

curl -X GET -H "Accept: application/xml" http://localhost:8080/article/view/1

This should return a response as follows:

<article><ID>1</ID><Title>Article 1</Title><Content>Article 1 body</Content></article>

As you can see, our request got a response in the XML format because we set the Accept header to application/xml.

Testing the Application

Since we’ve been using tests to create specifications for our route handlers and models, we should constantly be running them to ensure that the functions work as expected. Let’s now run the tests that we have written and see the results. In your project directory, execute the following command:

Executing this command should result in something similar to this:

=== RUN TestShowIndexPageUnauthenticated [GIN] 2022/05/11 - 11:33:20 | 200 | 429.084µs | | GET "/" --- PASS: TestShowIndexPageUnauthenticated (0.00s) === RUN TestGetAllArticles --- PASS: TestGetAllArticles (0.00s) PASS ok github.com/tomfern/semaphore-demo-go-gin	0.704s

As can be seen in this output, this command runs all the tests that we have written and, in this case, indicates that our application is working as we intend it to. If you take a close look at the output, you’ll notice that Go made HTTP requests in the course of testing the route handlers.

Continuous Integration for Go on Semaphore

Continuous Integration (CI) can test and build the application for us in a fast and clean environment. When we are ready to publish, Continuous Delivery (CD) can make the releases, secure in the knowledge that the code passed all our tests.

First, we have to get all the code in GitHub:

$ git init $ git remote add YOUR_REPOSITORY_URL $ git add -A $ git commit -m "initial commit" $ git push origin main

Add Semaphore to Your Project

Adding CI/CD to your project is completely free and takes only a few minutes:

1. Sign up for a free Semaphore account.
2. Click on Create new on the top navigation bar.

sem-version go 1.18

That’s it, on every push the CI pipeline will test and build the application.

Improving the Pipeline

The starter CI pipeline should work seamlessly without any additional setup. We can, however, make some improvements:

• Dependencies have to be downloaded every time. We can use a cache to keep them and speed up things.
• Testing and building are on the same job. We should split it in different jobs so it’s easier for us to later add more tests.

Use the Edit Workflow button to open the Workflow Builder:

The main elements of the builder are:

• Pipeline: A pipeline has a specific objective, e.g. testing. Pipelines are made of blocks that are executed from left to right in an agent.
• Agent: The agent is the virtual machine that powers the pipeline. We have three machine types to choose from. The machine runs an optimized Ubuntu 20.04 image with build tools for many languages.
• Block: blocks group jobs that can be executed in parallel. Jobs in a block usually have similar commands and configurations. Once all jobs in a block complete, the next block begins.
• Job: jobs define the commands that do the work. They inherit their configuration from their parent block.

We’ll make an improved version of the first block:

1. Click on the first block and change its name to: “Install dependencies”.
2. Below you’ll find the job, change its name to “Install” and type the following commands in the box:

sem-version go 1.18 export GO111MODULE=on export GOPATH=~/go export PATH=/home/semaphore/go/bin:$PATH checkout cache restore go mod vendor cache store

We’ve modified the block so it only downloads the Go dependencies:

• sem-version: a Semaphore built-in command to manage programming language versions. Semaphore supports most Go versions.
• checkout: another built-in command, checkout clones the repository and changes the current directory.
• go mod vendor: this a Go command that downloads the dependencies into the vendor directory, so they can be cached.
• cache: the cache commands provides read and write access to Semaphore’s cache, a project-wide storage for the jobs.

The first time that the jobs run, Go will download the dependencies and Semaphore will store them in the cache. For all the following runs, Semaphore will restore them and Go won’t need to download them again, thus speeding up the process considerably.

Testing with Semaphore

We expect our CI pipeline to be able to test the project and build a binary. We’ll add two more blocks for that:

1. Click on Edit Workflow
2. Use the +Add Block dotted line button to create a new block. Name the block “Test”
3. Open the Prologue section and type the following commands. The commands are executed before all jobs in the block:

sem-version go 1.18 export GO111MODULE=on export GOPATH=~/go export PATH=/home/semaphore/go/bin:$PATH checkout cache restore go mod vendor

Go microservice

1. Add the third block, let’s call it “Build”
2. Repeat the Prologue and Environment Variables steps as before.
3. Set the name of the job to “Build”
4. Type the following commands in the box:

go build -v -o go-gin-app artifact push project --force go-gin-app

The updated pipeline is now complete:

The artifact command we used in the build job uploads the binary into the project’s storage. To access it, use the Project Artifact button:

Semaphore has three separate artifact stores: Job, Workflow, and Project. For more information, check the artifacts doc.

Conclusion

In this tutorial, we created a new web application using Gin and gradually added more functionality. We used tests to build robust route handlers and saw how we can reuse the same code to render a response in multiple formats with minimal effort.

The code for the entire application is available in this Github repository.

Gin is easy to get started with-coupled with Go’s built-in functionality, its features make building high quality, well-tested web applications and microservices a breeze. If you have any questions or comments, feel free to post them below.

Originally published at https://semaphoreci.com on June 2, 2022.