ArtStation: A Week From Empty Repo to a Tested, Multi-GPU Texture Pipeline

QuickTile Studio turns a material into a seamless tile. Selling textures needs the materials themselves, at volume, in consistent styles. In the last week of February I built ArtStation, a private pipeline that generates tileable textures on my own GPUs, aimed at texture packs for the Unity Asset Store.

The repository was created on February 22. By the end of February 28:

Commits	38, including the initial commit
Version	v0.2.0 on February 27, v0.2.2 the same day
Prompt definitions	1,188 YAML files: 132 texture types in each of 9 styles
Working steps	base generation, img2img refinement, image captioning, LoRA training
Still stubbed	combinatorial final generation; Unity material preparation not started
Test cases in the repo	502 pytest, 204 vitest, 38 Playwright
GPU scheduling	one worker per Stable Diffusion server instance; two GPUs on the main machine

The test numbers are counts of test functions in the committed suite, not a run result. And this post is about the pipeline, not its output: generated images are deliberately not committed, so the repository has no record of how many it produced, and I'm not going to estimate one.

AI coding agents wrote most of the code under my direction (17 of the 38 commits carry an agent co-author line). I set the architecture, wrote the requirements, and reviewed what landed.

The shape: folders as the contract

The workflow had been proven by hand before the repository existed. The README records 12 prompt definitions validated in the Stylized style with a trained style LoRA. ArtStation automates that loop. Each step reads the previous step's folder and writes its own:

Prompts/<Style>/<type>.yaml
  -> Step 1  txt2img, tiling=true         Generated/Step1-Base/<Style>/<Type>/
  -> Step 2  img2img, denoise ~0.45       Generated/Step2-Denoised/<Style>/<Type>/
  -> Annotate  vision-model captions      .txt beside each training image
  -> Step 3  LoRA training (kohya_ss)     trained .safetensors
  -> Step 4  LoRA-enhanced final set      (stub at month end)
  -> Step 5  PBR maps, Unity materials    (planned)

Generation goes through the REST API of a local SDNext server. Captioning uses the Moondream vision model through Ollama. Training shells out to kohya_ss, because SDNext does not expose training over its API.

Folders as the interface meant any step could be re-run, inspected or replaced on its own, and the web UI added later needed no new contract: it browses the same tree.

Every image carries its own request

The first version wrote a .txt of the prompt next to each image. On February 23 that changed: steps 1 and 2 now write a .meta.json beside every image with three parts:

• request: the full API payload sent to SDNext
• response: the parsed generation info (seed, sampler, model)
• meta: style, texture type, run ID, timestamp

Two details mattered. The img2img input image is redacted from the stored payload, since a base64 copy would make every sidecar several megabytes. And .txt files became reserved for one purpose only, LoRA training captions, because kohya_ss reads captions from .txt files with the same name as the image. Two meanings for one file extension would have fed prompts into training as captions.

The caption format is fixed so training can rely on it:

<trigger_word>, <texture type>, top-down terrain, <Moondream description>

The first three tokens are constant, matching keep_tokens=3 in the training config.

Configuration as a resolution chain

Each value resolves through four layers, last one wins:

pipeline defaults -> style config.yaml -> preset -> CLI flags

Presets are named override sets (fast-iteration, high-quality, no-tile-explore, and a model test preset). Every step command accepts --preset and --override.

On top of that, pipeline.yaml has a per-machine section keyed by host name and deep-merged at load time. The same checkout runs on two machines with different drive layouts and GPU counts without a branch or a local edit.

Sharing GPUs between three model servers

Three model servers compete for the same memory: SDXL in SDNext, Moondream in Ollama, and kohya_ss during training. The decisions:

• Captioning unloads after every request. The Ollama service is configured with a keep-alive of 0, so the caption model does not stay resident next to SDXL.
• Checkpoint unloading was added, then reversed. On February 23, steps 1 and 2 began unloading the SDXL checkpoint when they finished. Five days later those calls were removed in the same commit that taught set_model to skip the reload when the requested checkpoint is already loaded. My reading of the trade-off: with a dedicated SDNext instance per GPU, a resident checkpoint is the cache, and reloading it for every job is the cost. Which is right depends on whether the GPU is shared; the per-GPU instance setup landed about an hour later the same day.
• One worker per GPU. The job queue spawns one async worker per SDNext instance. Each runs its blocking HTTP calls on a thread pool sized to the instance count, so a long generation never blocks the FastAPI event loop. Cancelling a job sends an interrupt to the instance running it, not to all of them.
• Visibility. nvidia-smi snapshots (VRAM, utilization, temperature) are logged at pipeline checkpoints and shown on the dashboard per GPU.

Errors that say what kind they are

On the second day the pipeline got structured logging and an error hierarchy:

class PipelineError(Exception): ...
class NetworkError(PipelineError): ...
class ConfigError(PipelineError): ...
class FileSystemError(PipelineError): ...
class TrainingError(PipelineError): ...
class OllamaError(PipelineError): ...

Every raise_for_status() in the SDNext client is wrapped into a NetworkError. A run ID is generated per run and injected into every log line across steps, and each service (pipeline, SDNext, kohya_ss) gets its own rotating log file. When a batch run fails hours in, the first question is which class of failure it was and which run it belonged to. Both are now a log filter.

Running as services, not terminals

A pipeline that only runs while a terminal is open isn't infrastructure. On February 27 the components became Windows services installed through NSSM, with an installer that:

• passes service arguments separately from the executable path, so install paths with spaces work
• guards the hosts-file append against a file with no trailing newline
• continues past a failed service, prints a per-service success and failure summary, and exits non-zero if anything failed
• auto-detects the Python executable instead of hard-coding it

That commit added 29 installer unit tests. The version moved to a single source in the Python package; the web frontend's version file is generated from it, and the frontend is rebuilt on every service restart, so the UI cannot report a stale version.

Taxonomy from the market

A texture pack is a taxonomy before it is a set of images. To check mine against what already sells, I added a Playwright scraper that extracts the content tree from Asset Store package pages. The pages use a virtualized list, so it expands every folder node and scrolls the container to force rendering before it reads the tree. It captured the content trees of three large competing texture collections for comparison, and the same commit added a Fabric and Textile category. The 132 Stylized texture types were then copied into eight more style folders as their starting point.

Tests, then the debt list

On February 28 the project got a test suite across all three layers: pytest for the pipeline and API, vitest for the Vue frontend, and Playwright end-to-end tests that mock the API with page.route(). One runner script runs all three and writes an HTML report.

The other habit worth keeping was a known-issues.md that names the debt precisely. One entry: the code read a combination key while every YAML file used combinations, so the setting was silently always empty and fell through to defaults. The loader now accepts both. Other entries call out a README schema that described the target design rather than the real files (now labelled as aspirational) and directory constants that nothing used. Writing them down meant an agent picking up the code later would not trust the wrong source.

What generalizes

Make every output carry its inputs. A sidecar with the exact request turns any image into a reproducible experiment, and it costs one file write.

Give each file type one meaning. Prompts and training captions both wanted .txt. Splitting them before training ran avoided a quiet data bug.

VRAM policy depends on topology. Unloading after each step is right when models share a GPU and wrong when each GPU has a dedicated server. I reversed the first choice the same day the per-GPU topology landed.

Write down the debt with the same precision as the code. A known-issues file that says what is wrong, where and why is what lets the next change, human or agent, start from the truth.