Integrations

Fyntune integrates with the major LLM providers and CI/CD systems. Select your integration type below.

OpenAI

Use the @track decorator with any openai.chat.completions.create call.

from fyntune import track
import openai

@track(feature="my-feature", version="v1")
def call_gpt(prompt: str) -> str:
    response = openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

Anthropic Claude

Works with the Anthropic Python SDK via the same @track decorator pattern.

from fyntune import track
import anthropic

client = anthropic.Anthropic()

@track(feature="my-feature", version="v1")
def call_claude(prompt: str) -> str:
    message = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}]
    )
    return message.content[0].text

LangChain

Wrap a LangChain chain or runnable with @track. Fyntune captures the final input and output text.

from fyntune import track
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

chain = ChatPromptTemplate.from_template("{input}") | ChatOpenAI()

@track(feature="rag-answer", version="v3")
def answer(question: str) -> str:
    result = chain.invoke({"input": question})
    return result.content

GitHub Actions

Add the Fyntune eval step to your workflow. Blocks merge on regression with an inline PR comment.

# .github/workflows/deploy.yml
name: Deploy with Fyntune eval
on: [push]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run Fyntune eval
        uses: fyntune-ai/eval-action@v2
        with:
          api_key: ${{ secrets.FYNTUNE_API_KEY }}
          block_on_regression: true
          notify: slack
          slack_webhook: ${{ secrets.SLACK_WEBHOOK }}

      - name: Deploy
        run: ./deploy.sh

GitLab CI

Use the Fyntune CI component in your .gitlab-ci.yml.

# .gitlab-ci.yml
include:
  - component: $CI_SERVER_FQDN/fyntune-ai/eval-component/run@v2

fyntune-eval:
  extends: .fyntune-eval
  variables:
    FYNTUNE_API_KEY: $FYNTUNE_API_KEY
    BLOCK_ON_REGRESSION: "true"
  stage: test

Vercel

Add the Fyntune deploy hook to your Vercel project to trigger evals on each deployment.

  1. Go to your Vercel project settings and add a Deploy Hook.
  2. Copy the hook URL into your Fyntune dashboard under Settings > Integrations > Vercel.
  3. Vercel deployments now automatically trigger a Fyntune eval run. If eval fails, the deployment is not blocked by Vercel — you'll receive a Slack/email alert with the regression details.

For automated blocking on Vercel preview deployments, use the Fyntune CLI in a pre-deployment GitHub Action instead (see GitHub Actions tab).