<p>I published my first Apify Actor at the end of 2024 after several years of building scrapers and automation tools in Go. Go is my most productive language, and most of my side projects are already written in it.</p>

<p>I knew Apify’s JavaScript and Python SDKs were the standard path. But my starting point was simple: I really wanted to build the Actor in Go. The side effects of that choice were lower memory usage, smaller Docker images, and full control over every API call. To support that workflow, I built a thin Go client on top of the Apify API, and this approach now underpins all the Actors in the private GitHub repository, where I manage them. The repo is private, but I’ll share the core patterns, trade-offs, and a few self-contained snippets you can reuse. I’ll include the snippets inline, and you can also browse them in this GitHub Gist.</p>

<p>This article is a practical walkthrough of that setup: how I structure Go-based Apify Actors, how I interact with the Apify API directly, and how I package small Docker images that reliably fit into the 128 MB Apify tier. I’ll use my Facebook (Meta) Ad Library Scraper Actor as a concrete example and reference real code from one of my Actors.</p><p>If you already know Apify and want a Go-first workflow with tight resource budgets, this is the approach that has worked well for me.</p><h2 id="project-context">Project context</h2><p>I keep all my Actors in a single monorepo. Each Actor lives under actors/<name>, while shared Apify-related logic sits in pkg/apify. This lets me reuse the same API client, logging conventions, and output logic across dozens of Actors without copy-pasting.</p><p>I chose to work directly with the Apify API for two main reasons:</p><ol><li>I wanted full control over runtime cost and memory usage. A thin client means no hidden buffers, no background workers, and no extra dependencies.</li><li>Go is my default language. I iterate faster in a language I already know well than by learning a new SDK surface.</li></ol><p>The downside is obvious: I own the edge cases and any API changes. I’ll come back to those trade-offs later.</p><h2 id="architecture-overview">Architecture overview</h2><h3 id="repository-layout">Repository layout</h3><p>This is a simplified version of the core structure:</p><pre>actors/ fbads/ cmd/fbads/ fbads.go .actor/ pkg/ apify/ client.go apify.go saver.go Dockerfile Makefile </pre><p>Each Actor has its own cmd/<actor> entrypoint and a package under actors/<actor> containing the business logic. Shared Apify interaction code lives in pkg/apify, which includes:</p><ul><ul><li>a small HTTP client for Apify API calls (client.go, apify.go)</li></ul><li>an output saver that writes to the Apify dataset or to a local JSON file (saver.go)</li><li>helpers for input parsing and key-value store access</li></ul><p>This keeps Actor-specific code small and makes it easy to roll out improvements across all Actors.</p><h3 id="the-dual-mode-runtime">The dual-mode runtime</h3><p>Every Actor runs in two modes:</p><ul><li>Local mode, for development and debugging</li><li>Apify mode, when running on the platform</li></ul><p>The Actor decides which mode it’s in by checking environment variables. I use ACTOR_RUN_ID as the switch:</p><pre>var apifyClient *apify.Client if os.Getenv("ACTOR_RUN_ID") == "" { // Local mode: read input from file or CLI, write JSON to a file } else { // Apify mode: read input from Apify KV store, write to dataset apifyClient, err = apify.NewActor(cfg.Debug) } </pre><p>This single check keeps both execution paths aligned and avoids the common problem where “local mode” slowly diverges from how the Actor behaves on Apify.</p><h2 id="building-a-go-actor-with-the-apify-api">Building a Go Actor with the Apify API</h2><h3 id="input-handling-and-schema-alignment">Input handling and schema alignment</h3><p>When running on Apify, I fetch the input directly from the key-value store. The client performs a simple GET request:</p><pre>func (c *Client) GetInput(ctx context.Context, v any) error { u := fmt.Sprintf("v2/key-value-stores/%s/records/INPUT", c.key) if _, err := c.do(ctx, "GET", u, nil, v); err != nil { return fmt.Errorf("apify: couldn't get input: %w", err) } return nil }</pre><p>The input schema maps directly to a Go struct. For facebook-ad-library-scraper, it looks like this:</p><pre>type Input struct { ProxyConfiguration apify.ProxyConfiguration `json:"proxyConfiguration"` MaxItems int `json:"maxItems"` Query string `json:"query"` Advertisers []string `json:"advertisers"` Country string `json:"country"` Category string `json:"category"` MediaType string `json:"mediaType"` MinDate string `json:"minDate"` MaxDate string `json:"maxDate"` ActiveStatus string `json:"activeStatus"` }</pre><p>In local mode, I reuse the same struct but load it from a JSON file or a raw JSON string. This keeps local tests tightly aligned with real Apify input and avoids surprises when deploying.</p><h3 id="output-handling-and-dataset-writes">Output handling and dataset writes</h3><p>For output, I push items to the default dataset when running on Apify and write to a local JSON file in local mode. This logic is centralized in pkg/apify/saver.go:</p><pre>func (s *Saver[T]) Save(ap *Client, output string, current []T) error { ctx, cancel := context.WithTimeout(context.Background(), time.Minute) defer cancel()

ARG ACTOR="" RUN set -eux;
if [ -z "$ACTOR" ]; then
if [ -n "$ACTOR_PATH_IN_DOCKER_CONTEXT" ]; then
ACTOR="$(basename "$ACTOR_PATH_IN_DOCKER_CONTEXT")";
else
echo "Set –build-arg ACTOR=<name> (or ACTOR_PATH_IN_DOCKER_CONTEXT)"; exit 1;
fi;
fi;
make build ACTOR="$ACTOR"

running image

FROM alpine WORKDIR /home COPY –from=builder /src/bin/app /bin/app

ENTRYPOINT [ "/bin/app" ] </code></pre><p>The final image doesn’t include the Go toolchain, source code, or build caches. It’s just the binary.</p>

<h3 id="makefile-and-reproducible-builds">Makefile and reproducible builds</h3><p>I use a Makefile to produce static, reproducible binaries with CGO disabled:</p><pre>GOOS=$$os GOARCH=$$arch GOARM=$$arm CGO_ENABLED=0 \ go build \ -a -x -tags netgo,timetzdata -installsuffix cgo -installsuffix netgo \ -ldflags " \ -X main.version=$(VERSION) \ -X main.commit=$(COMMIT_SHORT) \ -X main.date=$(shell date -u +'%Y-%m-%dT%H:%M:%SZ') \ " \ -o "$$out" \ ./actors/$(ACTOR)/cmd/$(ACTOR)</pre><p>The netgo tag and CGO_ENABLED=0 produce a fully static binary. The timetzdata tag embeds timezone data so I don’t need to install tzdata in the runtime image.</p><h2 id="memory-footprint-and-runtime-cost">Memory footprint and runtime cost</h2><p>My target is the 128 MB Apify tier. I design these Actors to stay comfortably under that limit:</p><ul><li>I avoid headless browsers unless unavoidable</li><li>I stream results in small batches instead of buffering everything</li><li>The runtime image is minimal and contains no extra libraries</li></ul><p>The first version of this setup routinely crossed 128 MB and was killed mid-run. The culprit was a slice accumulating parsed results before flushing to the dataset. Switching to smaller batches and clearing slices after each save brought peak memory usage back under 50 MB.</p>

<p>I use the memory graph in the Apify run stats as my feedback loop. If I see a spike, it’s almost always caused by buffering too much data or holding large strings longer than intended.</p><h2 id="debugging-and-local-workflow">Debugging and local workflow</h2><p>I run every Actor locally with a JSON input file before deploying. My usual flow is:</p><ol><li>Prepare an input JSON file matching the Apify input schema</li><li>Run the binary directly (no Docker) for fast iteration</li><li>Write output to a local JSON file and inspect it</li></ol><p>In Facebook (Meta) Ad Library Scraper, I store debug artifacts under a logs directory and validate input early. The dual-mode runtime is the biggest time saver: one codebase, two environments, and no debug-only branches that later drift.</p><h2 id="trade-offs-and-lessons-learned">Trade-offs and lessons learned</h2><p>This approach isn’t perfect:</p><ul><li>No SDK means I own the API surface. When Apify changes an endpoint, I update my client.</li><li>No SDK helpers. I re-implemented dataset writes, input parsing, retries, and PPE logic.</li><li>Docs matter more. I keep the Apify API docs open when adding features.</li></ul><p>For quick one-off scrapers, the official SDKs are still the better tool. For long-running, cost-sensitive Actors, this setup has paid off for me.</p><p>If I were starting today, I’d still build the same way, but I’d invest earlier in tests around the API client and add a small CI check against a mocked Apify API.</p><h2 id="conclusion">Conclusion</h2><p>Using Go and the Apify API directly has been the best setup for my workload. It keeps resource usage predictable, makes local debugging fast, and lets me run Actors in the smallest Apify tier without surprises.</p><p>If you’re a Go developer and want fine-grained control over how your Actors behave, I’d recommend trying this approach. Start with a single Actor, keep the client minimal, and only add abstractions when they earn their place.</p>