Golang notes no. 3

May 22, 2023

  • Organising Database Access

1. Global Variable

The main advantage of using a global variable is that it simplifies the access to the database connection pool throughout your application. Instead of passing the pool as a parameter to each function, you can directly access it from any part of your code.

However, it's important to consider the context and complexity of your application. If your application grows larger or your handlers have multiple dependencies beyond just the database connection pool, it's generally better to use dependency injection instead of relying heavily on global variables.

For a slight improvement to the global variable pattern, you can use the InitDB() function to initialize the connection pool. This keeps all the database-related code in a single package and prevents accidental mutation of the global variable by other packages at runtime. Additionally, during testing, you can reuse the InitDB() function to set up a connection pool for your test database.

By organizing your code in this way, you can maintain a cleaner separation of concerns and make your code more testable. It's always a good practice to evaluate the needs and complexity of your application and choose the appropriate approach for managing your database connections.

Another advantage of the modified global variable approach is that the db variable is not exported. This means that it cannot be accidentally mutated by other packages at runtime, ensuring better encapsulation and reducing the risk of unwanted side effects.

During testing, the InitDB() function can be utilized to initialize a connection pool specifically for your test database. By calling it from the TestMain() function before running your tests, you can ensure that your tests have a separate and controlled database environment.

Overall, while using a global variable for the database connection pool can simplify access and provide convenience for smaller applications, it's essential to consider the trade-offs and the potential challenges it may introduce as your application grows. For more complex applications, adopting dependency injection and separating concerns can lead to better maintainability, testability, and scalability.

Remember to evaluate the specific needs and context of your application before deciding on the most suitable approach for managing your database connections.

Suppose you have a web application that manages user profiles and stores their information in a database. You want to establish a global variable to hold the database connection pool.

In your models/models.go file, you can define the following:

var db *sql.DB

// InitDB initializes the global variable for the database connection pool.
func InitDB(dataSourceName string) error {
    var err error

  db, err = sql.Open("mysql", dataSourceName)
  if err != nil {
      return err
  }

  return db.Ping()
}

In your main.go file, you can then use the InitDB function to set up the global variable and establish the database connection pool:

func main() {
  err := models.InitDB("mysql://user:pass@localhost/userdb")
  if err != nil {
      log.Fatal(err)
  }

  http.HandleFunc("/profiles", profileHandler)
  http.ListenAndServe(":8080", nil)
}

By calling** models.InitDB("mysql://user:pass@localhost/userdb")** within the main function, you ensure that the global db variable is initialized with the database connection pool before starting the web server.

This approach provides a convenient way to access the database connection pool throughout your application. However, it's important to consider the implications and potential limitations, especially as your application grows. As mentioned before, for more complex applications, it's generally recommended to explore dependency injection and other patterns to ensure better modularity and testability.

Remember to adapt the approach based on your specific application requirements and architecture.

2. Dependency injection

One of the great things about this pattern is how clear and transparent it is. It allows us to easily see the dependencies our handlers rely on and the actual values they receive during runtime. Everything is explicitly defined in one place, the Env struct, which gives us a clear overview of the dependencies and their values when we initialize it in the main() function.Another advantage is that our unit tests for the handlers can be completely self-contained.

In general, this dependency injection approach works nicely under certain circumstances. It's especially useful when there is a common set of dependencies that all our handlers need to access. Additionally, if we have our HTTP handlers residing in one package while the database-related code is scattered across multiple packages, this approach can bring everything together in a neat and organized manner. Moreover, if we don't need to mock the database for testing purposes, this pattern fits the bill perfectly.

package main

import (
"database/sql"
"net/http"
"github.com/example/bookstore/models"
)

// Create a custom Env struct which holds a connection pool.
type Env struct {
Bookstore *models.Bookstore
}

func main() {
  // Create a new instance of Bookstore, passing in the database connection.
  bookstore := models.NewBookstore(getDBConnection())
  // Create an instance of Env containing the Bookstore instance.
  env := &Env{Bookstore: bookstore}

  // Use env.booksIndex as the handler function for the /books route.
  http.HandleFunc("/books", env.booksIndex)
  http.ListenAndServe(":3000", nil)

  }

  func getDBConnection() *sql.DB {
  // Initialize the connection pool.
  db, err := sql.Open("postgres", "postgres://user:pass@localhost/bookstore")
  if err != nil {
  log.Fatal(err)
  }
  return db
  }

  // booksIndex is the handler function for the /books route.
  func (env *Env) booksIndex(w http.ResponseWriter, r *http.Request) {
  // Access the Bookstore instance from the Env struct.
  bookstore := env.Bookstore
  // Use the bookstore to retrieve and display the list of books.
  books, err := bookstore.GetAllBooks()
  if err != nil {
    http.Error(w, "Internal Server Error", http.StatusInternalServerError)
    return
  }

  // Render the list of books in the HTTP response.
  // ...

}

In this example, we have a models package that contains a Book struct. In the main package, we introduce a custom Env struct that holds a connection to the database. We initialize the connection pool and create an instance of Env that contains the database connection.

By using dependency injection, we can pass the Env instance to our handler functions. In this case, we use the env.booksIndex function as the handler for the /books route. Inside the booksIndex function, we can access the Bookstore instance from the Env struct, which provides the necessary database operations for handling book-related requests.

This approach allows us to clearly see the dependencies our handler functions rely on, as they are explicitly defined in the Env struct. Additionally, it enables us to easily test our handlers by providing a custom Env instance with a mock database connection for unit testing purposes.

Remember, dependency injection provides a flexible and decoupled way of managing dependencies, making our code more modular and easier to test.

3. Wrapping the connection pool

This pattern brings clarity and readability to our handlers by making database calls concise and intuitive. In the case of our BookStore example, instead of writing env.books.All(), we can directly call env.books.All() in our handlers, making the code easier to understand.

In larger applications, the database layer often has more dependencies than just the connection pool. With this pattern, we can encapsulate all those dependencies within the custom BookModel type. This eliminates the need to pass multiple parameters to each database function, keeping our code clean and organized.

An interesting advantage of defining the database actions as methods on our custom BookModel type is that it opens up the possibility of using interfaces. By replacing references to BookModel with an interface, we can easily create mock implementations of BookModel for testing purposes. This allows us to isolate our unit tests and verify the behavior of our handlers without relying on a real database.

By wrapping the connection pool with a custom type and using dependency injection through the Env struct, we achieve a modular and flexible architecture. This approach is particularly useful when there is a common set of dependencies required by our handlers, when the database layer has additional dependencies beyond the connection pool, and when we want to mock the database during testing.

type Env struct {
	// Wrap the connection pool with a custom type that implements the interface
	// describing the methods needed by our handlers.
	books BookModel
}

// Handler code references the interface
func (env *Env) booksIndex(w http.ResponseWriter, r *http.Request) {
	bks, err := env.books.AllBooks()
	if err != nil {
		log.Print(err)
		http.Error(w, http.StatusText(500), 500)
		return
	}

	for _, bk := range bks {
		fmt.Fprintf(w, "%s, %s, %s, $%.2f\n", bk.Isbn, bk.Title, bk.Author, bk.Price)
	}
}

main.go

package main

import (
	"fmt"
	"log"
	"net/http"
)

// Custom type that wraps the connection pool and implements the required interface.
type BookModel struct {
	db *sql.DB
}

// Method on BookModel that retrieves all books from the database.
func (bm *BookModel) AllBooks() ([]models.Book, error) {
	// Implementation of retrieving books from the database using bm.db.
	// ...
}

func main() {
	db, err := sql.Open("postgres", "postgres://user:pass@localhost/bookstore")
	if err != nil {
		log.Fatal(err)
	}

	env := &Env{
		books: &BookModel{db: db},
	}

	http.HandleFunc("/books", env.booksIndex)
	http.ListenAndServe(":3000", nil)
}

main_test.go

package main

import (
	"fmt"
	"log"
	"net/http"
	"net/http/httptest"
	"testing"
)

// Mock implementation of the BookModel interface for testing.
type mockBookModel struct{}

func (m *mockBookModel) AllBooks() ([]models.Book, error) {
	var bks []models.Book

	bks = append(bks, models.Book{"978-1503261969", "Emma", "Jayne Austen", 9.44})
	bks = append(bks, models.Book{"978-1505255607", "The Time Machine", "H. G. Wells", 5.99})

	return bks, nil
}

func TestBooksIndex(t *testing.T) {
	rec := httptest.NewRecorder()
	req, _ := http.NewRequest("GET", "/books", nil)

	env := &Env{
		books: &mockBookModel{},
	}

	http.HandlerFunc(env.booksIndex).ServeHTTP(rec, req)

	expected := "978-1503261969, Emma, Jayne Austen, $9.44\n978-1505255607, The Time Machine, H. G. Wells, $5.99\n"
	if expected != rec.Body.String() {
		t.Errorf("\n...expected = %v\n...obtained = %v", expected, rec.Body.String())
	}
}

4. Request context

Using the request context to store values is intended for passing data between different processes and APIs, rather than for passing optional parameters to functions.

When using the request context, it's important to limit its usage to storing data that is specific to an individual request and is no longer needed once the request is complete. It is not meant to be used for storing long-lived dependencies like connection pools, loggers, or template caches.

However, this pattern has some drawbacks that we should be aware of:

  • Retrieving the connection pool from the context requires type assertion and error handling, which adds verbosity to the code and eliminates the compile-time type safety that other approaches provide.
  • Unlike dependency injection patterns, it's not immediately clear what dependencies a function has just by looking at its signature. Instead, we need to examine the code to see what it retrieves from the request context. While this may not be a problem in small applications, it can become challenging in larger and unfamiliar codebases.
  • This usage of the request context is not considered idiomatic in Go. It goes against the advice provided in the official Go documentation, which means that other Go developers might find this pattern surprising or unfamiliar.

main.go

package main

import (
	"context"
	"database/sql"
	"fmt"
	"log"
	"net/http"

	"bookstore/models"

	_ "github.com/lib/pq"
)

// Middleware function that injects the database connection pool into the request context.
func injectDB(db *sql.DB, next http.HandlerFunc) http.HandlerFunc {
	return func(w http.ResponseWriter, r *http.Request) {
		ctx := context.WithValue(r.Context(), "db", db)
		next.ServeHTTP(w, r.WithContext(ctx))
	}
}

func main() {
	db, err := sql.Open("postgres", "postgres://user:pass@localhost/bookstore")
	if err != nil {
		log.Fatal(err)
	}

	// Wrap the booksIndex handler with the injectDB middleware,
	// passing in the new context.Context with the connection pool.
	http.Handle("/books", injectDB(db, booksIndex))
	http.ListenAndServe(":3000", nil)
}

func booksIndex(w http.ResponseWriter, r *http.Request) {
	// Retrieve the request context containing the database connection pool.
	ctx := r.Context()

	// Pass the request context to the database layer.
	bks, err := models.AllBooks(ctx)
	if err != nil {
		log.Print(err)
		http.Error(w, http.StatusText(500), 500)
		return
	}

	for _, bk := range bks {
		fmt.Fprintf(w, "%s, %s, %s, £%.2f\n", bk.Isbn, bk.Title, bk.Author, bk.Price)
	}
}

bookstore/models.go

package models

import (
	"context"
	"database/sql"
	"errors"
)

type Book struct {
	Isbn   string
	Title  string
	Author string
	Price  float32
}

func AllBooks(ctx context.Context) ([]Book, error) {
	// Retrieve the connection pool from the context. Since the Value() method returns an interface{},
	// we need to type assert it into a *sql.DB before using it.
	db, ok := ctx.Value("db").(*sql.DB)
	if !ok {
		return nil, errors.New("failed to retrieve database connection pool from context")
	}

	rows, err := db.Query("SELECT * FROM books")
	if err != nil {
		return nil, err
	}
	defer rows.Close()

	var bks []Book

	for rows.Next() {
		var bk Book
		err := rows.Scan(&bk.Isbn, &bk.Title, &bk.Author, &bk.Price)
		if err != nil {
			return nil, err
		}
		bks = append(bks, bk)
	}

	if err = rows.Err(); err != nil {
		return nil, err
	}

	return bks, nil
}

In this example, we utilize the concept of request context to pass the database connection pool to the handler function. The injectDB middleware injects the connection pool into the request context, allowing it to be accessed within the booksIndex handler.

By passing the request context to the AllBooks function in the models package, we can retrieve the database connection pool from the context and perform database operations. This approach ensures that each request has its own isolated database connection.

The use of request context in this manner provides a way to manage and pass dependencies specific to a particular request. It allows for better encapsulation and decoupling of components, making the code more modular and testable.