Reference Counted Duplicator in Golang

A Comprehensive Guide to Implementing a Reference Counting Pattern in Go

Highlights

Thread Safety: Utilizing atomic operations guarantees thread-safe updates to reference counts, ensuring safe usage in concurrent environments.
Resource Management: The duplicator copies objects and manages their lifecycles, guaranteeing that resources are freed only when no longer used.
Custom Release Functions: Integrate your custom cleanup or release logic for handling external or non-memory resources effectively.

In Golang, the garbage collector generally takes care of memory management automatically through a tracing algorithm, but specific scenarios require more control over objects’ lifetimes. One solution is to implement a reference counted duplicator. In this approach, each object is wrapped such that its usage is tracked via a reference count; the object is cleaned up only when the reference count drops to zero.

Fundamental Concepts

What is Reference Counting?

Reference counting is a memory management technique which involves maintaining a counter for each object that indicates how many references exist to that object. When the number of references falls to zero, the object is deemed no longer in use, and its associated resources can be safely deallocated. This method is especially beneficial when dealing with external resources (like file handles or network connections) where you want to ensure that cleanup occurs at the appropriate moment.

Thread Safety and Atomic Operations

Given Go’s inherent support for concurrent routines, any operation that adjusts reference counts must be thread-safe. This is typically achieved by using the atomic functions provided by the sync/atomic package. Atomic operations are crucial for incrementing and decrementing reference counts, ensuring consistency when multiple goroutines interact with the same object.

Duplication vs. Deep Copying

A major challenge in object duplication in Go is correctly replicating an object especially when deep copying is required. Using reflection can help traverse and copy the structure, but note that it will not work with all types of fields (particularly unexported ones or channels and functions). Therefore, you may need to tailor the duplication logic to your specific needs. In our duplicator design, the original object’s value is duplicated to produce a fresh instance with its own reference count—a strategy that allows multiple consumers to share the underlying data while managing its lifecycle independently.

Designing the Reference Counted Duplicator

Data Structures

The core data structure is a wrapper that stores the victim object alongside a reference count. We can define this structure to accept any data type using Go’s generics, ensuring that the mechanism is not tied to a specific type. Additionally, a custom cleanup function can be provided to release any non-memory resources when the count hits zero.

Example: RefCountedObject Structure

The following structure demonstrates a generic wrapper that encapsulates an object of any type T. It holds an atomic reference count, a mutex for safe access, and a cleanup function to be executed upon final release:


// Define the generic structure for a reference counted object.
package refcount

import (
	"sync"
	"sync/atomic"
)

// RefCountedObject wraps any object with reference counting.
type RefCountedObject[T any] struct {
	value     T
	refCount  int64
	onRelease func(T)
	mutex     sync.RWMutex
}

// New creates a new reference counted object with an initial count of 1.
func New[T any](value T, releaseFunc func(T)) *RefCountedObject[T] {
	return &RefCountedObject[T]{
		value:     value,
		refCount:  1,
		onRelease: releaseFunc,
	}
}

Reference Count Manipulation

The two core methods are to increase (Acquire) and decrease (Release) the reference count. When a new reference is made, the count is increased atomically. Conversely, when a reference is no longer required, the count is decreased. When the count reaches zero, the object’s resources are released by calling the custom cleanup function.

Implementation of Acquire and Release


// Increment the reference count.
func (r *RefCountedObject[T]) Acquire() *RefCountedObject[T] {
	atomic.AddInt64(&r.refCount, 1)
	return r
}

// Release decrements the count and performs cleanup if it reaches zero.
func (r *RefCountedObject[T]) Release() {
	if atomic.AddInt64(&r.refCount, -1) == 0 {
		r.mutex.Lock()
		defer r.mutex.Unlock()
		if r.onRelease != nil {
			r.onRelease(r.value)
		}
	}
}

With these functions, you can safely spread references among different consumers or goroutines, ensuring that the object persists as long as needed.

Duplicating Objects

In many cases, you might wish to create an independent duplicate of an object. The duplicator must create a copy of the underlying data while instantiating its own reference counter. It ensures that two separate objects can manage their lifetimes independently even when they have identical content. A straightforward approach involves copying the value and reinitializing the counter.

The Duplicate Method

The Duplicate method creates a new instance of the object with the same content and a fresh reference count. Though simple, special care should be taken if your object has nested structures, slices, or maps. A deep copy or dedicated copy logic might be required.


// Duplicate returns a new object with the same underlying value.
func (r *RefCountedObject[T]) Duplicate() *RefCountedObject[T] {
	r.mutex.RLock()
	defer r.mutex.RUnlock()
	// Create a new reference counted object with a fresh counter.
	return New(r.value, r.onRelease)
}

A Complete Example: Combining the Concepts

Putting It All Together

The following comprehensive example demonstrates the complete implementation of a reference counted duplicator. It defines a RefCountedObject, performs duplications, and manages object lifetimes with explicit Acquire and Release calls. In this example, we simulate releasing a non-memory resource by printing a message when an object’s reference count drops to zero.


package main

import (
	"fmt"
	"sync"
	"sync/atomic"
)

// RefCountedObject wraps any type with reference counting and a release function.
type RefCountedObject[T any] struct {
	value     T
	refCount  int64
	onRelease func(T)
	mutex     sync.RWMutex
}

// New creates a new RefCountedObject with an initial ref count of 1.
func New[T any](value T, releaseFunc func(T)) *RefCountedObject[T] {
	return &RefCountedObject[T]{
		value:     value,
		refCount:  1,
		onRelease: releaseFunc,
	}
}

// Acquire increments the reference count atomically.
func (r *RefCountedObject[T]) Acquire() *RefCountedObject[T] {
	atomic.AddInt64(&r.refCount, 1)
	return r
}

// Release decrements the reference count and calls onRelease when it reaches 0.
func (r *RefCountedObject[T]) Release() {
	if atomic.AddInt64(&r.refCount, -1) == 0 {
		r.mutex.Lock()
		defer r.mutex.Unlock()
		if r.onRelease != nil {
			r.onRelease(r.value)
		}
	}
}

// Duplicate creates a new instance with the same underlying value and reset ref count.
func (r *RefCountedObject[T]) Duplicate() *RefCountedObject[T] {
	r.mutex.RLock()
	defer r.mutex.RUnlock()
	return New(r.value, r.onRelease)
}

// Value returns the wrapped object value.
func (r *RefCountedObject[T]) Value() T {
	r.mutex.RLock()
	defer r.mutex.RUnlock()
	return r.value
}

type Resource struct {
	ID   int
	Data string
}

// main demonstrates reference counting with a duplicator.
func main() {
	// Custom cleanup function.
	releaseFunc := func(r Resource) {
		fmt.Printf("Cleaning up resource with ID: %d\n", r.ID)
	}

	// Create a resource and wrap it with a reference counted object.
	original := New(Resource{ID: 42, Data: "Hello, World!"}, releaseFunc)
	fmt.Printf("Initial resource: %+v with ref count: %d\n", original.Value(), atomic.LoadInt64(&original.refCount))

	// Duplicate the original object.
	duplicate := original.Duplicate()
	fmt.Printf("Duplicated resource: %+v with ref count: %d\n", duplicate.Value(), atomic.LoadInt64(&duplicate.refCount))

	// Simulate multiple references.
	// Increase reference count manually.
	original.Acquire()
	fmt.Printf("After acquiring, original ref count: %d\n", atomic.LoadInt64(&original.refCount))

	// Release one reference from the original.
	original.Release()
	fmt.Printf("After releasing one ref, original ref count: %d\n", atomic.LoadInt64(&original.refCount))
	
	// Release the duplicate, which should trigger cleanup if no further references exist.
	duplicate.Release()
  
	// Final release for the original reference.
	original.Release()
}

Understanding the Workflow via a Table

The following table summarizes the operations performed by our reference counted duplicator:

Operation	Description	Effect on Reference Count
New	Creation of a new RefCountedObject with initial count 1	1
Acquire	Increment the reference count when a new reference is needed	+1
Duplicate	Create a new instance with the same value and its own counter	New object has count 1
Release	Decrement the reference count; trigger cleanup at zero	-1 (Cleanup if count is 0)

Advanced Considerations

Handling Circular References

One serious pitfall with reference counting is the possibility of creating circular references. This occurs when two or more objects reference each other, meaning their counts never diminish to zero. In such cases, even if no external references are holding onto these objects, they remain in memory. To manage this, consider implementing weak references or utilizing another higher-level garbage collection mechanism to detect and break cycles.

Performance and Overhead

While the atomic operations provide the necessary safety in concurrent environments, they inevitably add some overhead. This overhead is generally minimal compared to the cost of most I/O or computation tasks. However, for performance-critical applications, it is worth analyzing whether the careful management offered by reference counting is warranted compared to Go’s default garbage collection. In scenarios where frequent copies occur, consider using sync.Pool for object reuse to mitigate resource overhead.

Real-World Applications

Reference counting is particularly useful when working with external resources that require deterministic finalization—such as file descriptors, network connections, or large data objects that must be cleaned up quickly. Additionally, in situations where objects must be retained across multiple modules or complex systems (like plugin architectures or shared caches), reference counting ensures that objects are not prematurely cleaned up.

Integrating with Go's Garbage Collector

It is important to remember that Go’s garbage collector does not employ reference counting; it uses a modern tracing algorithm that works well for the vast majority of applications. Reference counting should be reserved for those special cases where manual control of object lifetimes is explicitly required. In cases where a reference counted approach is implemented alongside the conventional GC, care must be taken to avoid double free scenarios or memory retention due to mismanaged reference counts.

Conclusion

Implementing a reference counted duplicator in Golang provides robust control over object lifetimes, especially in concurrent and resource-sensitive environments. By combining atomic operations for thread-safe reference count manipulation with the flexibility of object duplication, developers can ensure that resource cleanup is handled precisely when it is safe to do so.

The provided example demonstrates the power of this approach with a generic structure that supports any data type, a custom cleanup function for targeted resource management, and mechanisms to duplicate objects safely. While there are inherent challenges—such as the need to prevent circular references and manage performance overhead—the benefits of deterministic resource management in critical sections of code are substantial.

In practice, this reference counted duplicator design can be adapted and extended to serve various purposes in production-grade applications, particularly where precise control over resource lifetimes is necessary. Whether you need to manage large in-memory datasets, ensure the timely release of system resources, or handle concurrent operations in a multi-threaded environment, this approach to reference counting and object duplication in Go offers a flexible solution.

References

Reference Counted Pool - Hydrogen18
Are unsafe pointers reference counted? - StackOverflow
gvisor.dev/gvisor/pkg/refs - Go Packages
Show HN: A pure reference counting GC in Go - Hacker News