GitHub Models and other LLM inference services

27-08-2024

• https://github.com/marketplace/models
  • github models is being trialed in beta atm, allows you to experiment with different closed and open source models with inference running on azure, they have a playground to experiment with, plus you can spin up a codespace or work locally using free api creds, or use your own creds if you want greater rate limits and control
• some other SAAS inference offerings that are quite fun (the open source stuff is fine for tinkering but can be painful)
  • https://replicate.com/
  • https://fireworks.ai/ (which is used by https://www.cursor.com/))
  • https://lambdalabs.com/ (for compute)