โ† AI Tools
Cloud AIBeginner

Replicate

A cloud inference platform that allows you to instantly deploy open-source AI models as REST APIs.

Replicate is a cloud AI model hosting platform co-founded in 2019 in the United States by Ben Firshman and Andreas Jansson. In short, it can be described as the "Stripe API for open-source AI models." It allows you to call 10,000+ models like Llama, Stable Diffusion XL, Whisper, MusicGen, and SAM via a single REST API line without using a GPU, and charges you by the second for the GPU usage (e.g., SDXL 1 image ~/bin/zsh.0023, Llama-3 70B 1k tokens ~/bin/zsh.001). Existing cloud AI solutions had limitations such as (1) directly deploying models on general clouds like AWS, GCP, and Azure, which creates infrastructure and MLOps burdens; (2) using OpenAI or Anthropic APIs, which limits model selection; and (3) using the Hugging Face Inference API, which only supports a limited number of models and restricts large LLMs. Replicate standardizes the workflow of packaging models with "Cog," an open-source container standard, allowing users to visit the model page, "Run" or call the API, and receive the results via webhook/streaming. It's like "Vercel standardized the deployment of front-end applications, and Replicate standardizes the deployment of AI models." The model page clearly displays the input/output schema, example results, price, and average inference time, reducing integration efforts. From a biotechnology researcher's perspective, it enables (1) calling protein structure prediction models like ESMFold and OpenFold without a GPU, (2) automatically generating paper figures and research illustrations with Stable Diffusion XL, (3) performing batch STT on conference and interview recordings with Whisper, (4) generating lecture background music with MusicGen and AudioCraft, and (5) deploying custom models (by packaging them with Cog, allowing you to host your own models on Replicate). If your monthly usage is low, it is more cost-effective than Hugging Face Spaces or the Inference API, and if your usage is high, you can fix the cost with dedicated deployment (reserved GPU). Using it in parallel with Ollama and Hugging Face creates strong synergy. For example, you can use a phased expansion approach like "use local Ollama for rapid prototyping, then burst to Replicate when dealing with large models or traffic spikes, and finally use dedicated deployment for stable operation." Alternatively, a standard pattern is to "discover models on HF, deploy them to Replicate using Cog, and then call the API from your own application.

๐Ÿ’ป System Requirements

๐Ÿง RAM

0 GB when using only cloud inference (sufficient with just a browser or laptop). 16 GB or more when packaging your own model with Cog and performing local testing.

๐ŸŽฎVRAM

0 GB for cloud inference / 8-48 GB for Cog local testing, depending on the model size. Replicate server-side GPUs are automatically allocated as A100 or H100.

๐Ÿ’พStorage

0 GB for cloud use / 10-50 GB for Cog local builds (Docker image). Result files are automatically hosted on the Replicate CDN for 7 days (for download or webhook processing).

โšก Installation

# Python SDK
pip install replicate

# 1. Set environment variables (API token)
export REPLICATE_API_TOKEN=r8_xxx  # Obtain from replicate.com after signing up

# 2. Call a model (e.g., Llama-3 70B)
import replicate
output = replicate.run(
    "meta/meta-llama-3-70b-instruct",
    input={"prompt": "What is the difference between AlphaFold and ESMFold in protein structure prediction?", "max_tokens": 500}
)
print("".join(output))

# 3. Generate an image (Stable Diffusion XL)
output = replicate.run(
    "stability-ai/sdxl:39ed52f2a78e934b3ba6e2a89f5b1c712de7dfea535525255b1aa35c5565e08b",
    input={"prompt": "protein structure visualization, scientific illustration"}
)
# output[0] = URL of the generated image

# 4. Speech-to-text (Whisper)
output = replicate.run(
    "openai/whisper:91ee9c0c3df30478510ff8c8a3a545add1ad0259ad3a9f78fba57fbc05ee64f7",
    input={"audio": open("interview.mp3", "rb")}
)

# 5. Deploy your own model with Cog
pip install cog
# Create cog.yaml and predict.py, then
cog push r8.im/myusername/my-bio-model

# 6. Call the REST API directly (without Python)
curl -X POST https://api.replicate.com/v1/predictions -H "Authorization: Bearer " -d '{"version":"...","input":{"prompt":"..."}}
๐Ÿ“„ Official Docs๐Ÿ™ GitHub

๐Ÿ“ Update Notes

No update notes yet.

๐Ÿงช Related Code of Life

No related Code of Life posts yet.