It is compatible with the OpenAI Specification:
from openai import OpenAI
IONOS_API_TOKEN = ""
openai = OpenAI(
api_key=IONOS_API_TOKEN, base_url="https://openai.inference.de-txl.ionos.com/v1"
)
completion = openai.chat.completions.create(
model="meta-llama/Meta-Llama-3.1-8B-Instruct",
messages=[
{
"role": "user",
"content": "Wie ist das Wetter meistens in Deutschland?",
}
],
max_completion_tokens=400,
temperature=0,
top_p=0.1,
stream=True,
)
for chunk in completion:
print(chunk.choices[0].delta.content, end="")
Models
Chat models
See usage: Chat Models IONOS Cloud
OCR
https://docs.ionos.com/cloud/ai/ai-model-hub/models/ocr-models
LightOnOCR-2-1B
How to use the LightOnOCR model with the openai SDK: https://docs.ionos.com/cloud/ai/ai-model-hub/how-tos/pdf-to-text