Training Example: Comet – Review the Data, Give Your Score & Compare to the Real AI Evaluation

[H1] The Fastest Path to Agents That Work
Opik connects observability to action, automatically turning trace data and eval results into code fixes. Your agent keeps evolving and doesn’t make the same mistake twice.
Trusted by over 150,000 developers and thousands of companies
[IMG: AssemblyAI logo]
[IMG: Natwest logo]
[IMG: Stellantis logo]
[IMG: Uber Logo]
[IMG: zencoder logo]
[IMG: Netflix Logo]
[IMG: Autodesk logo]
[IMG: Etsy logo]
[IMG: Stability Ai logo]
[IMG: Mobileye logo]

[IMG: AssemblyAI logo]
[IMG: Natwest logo]
[IMG: Stellantis logo]
[IMG: Uber Logo]
[IMG: zencoder logo]
[IMG: Netflix Logo]
[IMG: Autodesk logo]
[IMG: Etsy logo]
[IMG: Stability Ai logo]
[IMG: Mobileye logo]
19,000+
Github Stars
150,000+
Users
10,000+
Teams
[H2] Log every step your agent takes
Traces give you total LLM observability to visualize and understand what’s happening across complex GenAI systems, from context retrieval and tool selection to user feedback scores and more.
Try Opik free
[H2] Annotate & debug individual traces
Review your traces to label what’s working, what’s not, and pinpoint where to iterate and improve. Invite SMEs to collaborate on human review directly inside the platform.
Try Opik free
[H2] Evaluate performance at scale
Auto-score large sets of traces with 30+ LLM-as-a-judge metrics for answer relevance, context precision, hallucination detection, and more — or try Opik’s new Test Suites for a simplified pass/fail workflow.
Try Opik free
[H2] Iterate & improve with Ollie
Ollie, Opik’s powerful built-in coding agent, analyzes your traces and test outcomes, identifies fixes, and writes them directly to your own agent’s codebase, with version control and regression testing.
Try Opik free
[H2] Monitor & manage agents in production
Opik extends observability and online evaluation across your agent’s production footprint to help meet governance requirements, track model costs, and ensure consistent performance in front of real users.
Learn more
Try Opik Free
Get Demo
“LLMs are black boxes. We don’t know what is going on inside them. We needed a solution that allowed us to see how our models behaved, and Opik gives us the ability to understand what went wrong, and share that with the team to debug and iterate faster.”
DMITRII KRASNOV
ENGINEERING MANAGER, ZENCODER
Trusted by the most innovative AI teams
[IMG: AssemblyAI logo]
[IMG: Natwest logo]
[IMG: Stellantis logo]
[IMG: Uber Logo]
[IMG: zencoder logo]
[IMG: Netflix Logo]
[IMG: Autodesk logo]
[IMG: Etsy logo]
[IMG: Stability Ai logo]
[IMG: Mobileye logo]

[IMG: AssemblyAI logo]
[IMG: Natwest logo]
[IMG: Stellantis logo]
[IMG: Uber Logo]
[IMG: zencoder logo]
[IMG: Netflix Logo]
[IMG: Autodesk logo]
[IMG: Etsy logo]
[IMG: Stability Ai logo]
[IMG: Mobileye logo]
[H2] The Opik Difference
Not all GenAI observability and evaluation platforms are built the same. Opik is both truly open source, and powered by Comet’s enterprise-grade infrastructure for reliable, trustworthy performance at scale.
[H3] Log Thousands of LLM Traces, Fast
Traces appear in the Opik platform ready for debugging almost instantly — even at high volumes.
[H3] Enterprise-Grade Reliability & Security
Opik is backed by the Comet platform and built to the standards of the world’s largest organizations.
[H3] Flexible Hosting & Deployment Options
Self-host the OSS version, try Opik in the cloud, or talk to us about custom deployment options.
[H2] Easy Integration
Add just a few lines of code to your project and automatically start tracking LLM app and agent activity with Opik, or code, hyperparameters, model predictions, and more with Comet’s MLOps platform.
Try Opik Cloud
View on GitHub
Opik LLM Evaluation
[IMG: any-framework-icon.svg]
Any LLMfrom opik import track
@track
def llm_chain(user_question):
context = get_context(user_question)
response = call_llm(user_question, context)
return response
@track
def get_context(user_question):
# Logic that fetches the context, hard coded here
return ["The dog chased the cat.", "The cat was called Luky."]
@track
def call_llm(user_question, context):
# LLM call, can be combined with any Opik integration
return "The dog chased the cat Luky."
response = llm_chain("What did the dog do ?")
print(response)Copy
[IMG: image-1.png]
LlamaIndexfrom llama_index.core import VectorStoreIndex, global_handler, set_global_handler
from llama_index.core.schema import TextNode
# Configure the Opik integration
set_global_handler("opik")
opik_callback_handler = global_handler
node1 = TextNode(text="The cat sat on the mat.", id_="1")
node2 = TextNode(text="The dog chased the cat.", id_="2")
index = VectorStoreIndex([node1, node2])
# Create a LlamaIndex query engine
query_engine = index.as_query_engine()
# Query the documents
response = query_engine.query("What did the dog do ?")
print(response)Copy
[IMG: langchain-1-1.png]
LangChainfrom langchain_openai import ChatOpenAI
from opik.integrations.langchain import OpikTracer
# Initialize the tracer
opik_tracer = OpikTracer()
# Create the LLM Chain using LangChain
llm = ChatOpenAI(temperature=0)
# Configure the Opik integration
llm = llm.with_config({"callbacks": [opik_tracer]})
llm.invoke("Hello, how are you?")Copy
[IMG: OpenAI-code.png]
OpenAIfrom openai import OpenAI
from opik.integrations.openai import track_openai
openai_client = OpenAI()
openai_client = track_openai(openai_client)
response = openai_client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[
{"role": "user", "content": "Hello, world!"}
]
)Copy
ML Experiment Management
[IMG: pytorch-icon-no-fill.svg]
Pytorchfrom comet_ml import Experiment
import torch.nn as nn
# 1. Define a new experiment
experiment = Experiment(project_name="YOUR PROJECT")
# 2. Create your model class
class RNN(nn.Module):
#... Define your Class
# 3. Train and test your model while logging everything to Comet
with experiment.train():
# ...Train your model and log metrics
experiment.log_metric("accuracy", correct / total, step = step)
# 4. View real-time metrics in CometCopy
[IMG: pytorch-lightning-icon.svg]
Pytorch Lightningfrom pytorch_lightning.loggers import CometLogger
# 1. Create your Model
# 2. Initialize CometLogger
comet_logger = CometLogger()
# 3. Train your model
trainer = pl.Trainer(
logger=[comet_logger],
# ...configs
)
trainer.fit(model)
# 4. View real-time metrics in CometCopy
[IMG: hugging-face-icon-no-fill.svg]
Hugging Facefrom comet_ml import Experiment
from transformers import Trainer
# 1. Define a new experiment
experiment = Experiment(project_name="YOUR PROJECT")
# 2. Train your model
trainer = Trainer(
model = model,
# ...configs
)
trainer.train()
# 3. View real-time metrics in CometCopy
[IMG: keras-icon.png]
Kerasfrom comet_ml import Experiment
from tensorflow import keras
# 1. Define a new experiment
experiment = Experiment(project_name="YOUR PROJECT")
# 2. Define your model
model = tf.keras.Model(
# ...configs
)
# 3. Train your model
model.fit(
x_train, y_train,
validation_data=(x_test, y_test),
)
# 4. Track real-time metrics in CometCopy
[IMG: tensorflow-icon-no-fill.svg]
TensorFlowfrom comet_ml import Experiment
import tensorflow as tf
# 1. Define a new experiment
experiment = Experiment(project_name="YOUR PROJECT")
# 2. Define and train your model
model.fit(...)
# 3. Log additional model metrics and params
experiment.log_parameters({'custom_params': True})
experiment.log_metric('custom_metric', 0.95)
# 4. Track real-time metrics in CometCopy
[IMG: scikit-learn-icon-no-fill.svg]
Scikit-learnfrom comet_ml import Experiment
import tree from sklearn
# 1. Define a new experiment
experiment = Experiment(project_name="YOUR PROJECT")
# 2. Build your model and fit
clf = tree.DecisionTreeClassifier(
# ...configs
)
clf.fit(X_train_scaled, y_train)
params = {...}
metrics = {...}
# 3. Log additional metrics and params
experiment.log_parameters(params)
experiment.log_metrics(metrics)
# 4. Track model performance in CometCopy
[IMG: xgboost-icon-no-fill.png]
XGBoostfrom comet_ml import Experiment
import xgboost as xgb
# 1. Define a new experiment
experiment = Experiment(project_name="YOUR PROJECT")
# 2. Define your model and fit
xg_reg = xgb.XGBRegressor(
# ...configs
)
xg_reg.fit(
X_train,
y_train,
eval_set=[(X_train, y_train), (X_test, y_test)],
eval_metric="rmse",
)
# 3. Track model performance in CometCopy
[IMG: any-framework-icon.svg]
Any Framework# Utilize Comet in any environment
from comet_ml import Experiment
# 1. Define a new experiment
experiment = Experiment(project_name="YOUR PROJECT")
# 2. Model training here
# 3. Log metrics or params over time
experiment.log_metrics(metrics)
#4. Track real-time metrics in CometCopy
[H2] An End-to-End Model Evaluation Platform
Comet’s end-to-end model evaluation platform for developers focuses on shipping AI features, including open source LLM tracing, ML unit-testing, evaluations, experiment tracking and production monitoring.
[H3] Opik: Log & Evaluate Your Application’s LLM Calls
Opik provides comprehensive LLM observability so you can confidently test, debug, and monitor your GenAI apps and agents, from application-level unit testing down to individual system prompts and user inputs.
[H3] Opik: Optimize Prompts & Agentic Systems
With your application’s LLM calls and responses logged, you can bring in expert reviewers for annotation, score using built-in eval metrics, and even automate prompt engineering for complex multi-step agents.
[H3] MLOps: Track & Compare Model Training Runs
Comet Experiment Management gives you the tools to ensure your models are explainable and reproducible, with custom visualizations, model versioning, dataset management, production monitoring, and more.
[H3] ML Model Production Monitoring
Deploy your optimized models with confidence, ensure regulatory compliance, and catch and fix issues like data drift before they start to affect your end-user experience.
[H2] Built for Enterprise, Driven by Community
Comet’s end-to-end evaluation platform is trusted by innovative data scientists, ML practitioners, and engineers in the most demanding enterprise environments.
“Comet has aided our success with ML and serves to further ML development within Zappos.”
KYLE ANDERSON
DIRECTOR OF SOFTWARE ENGINEERING
“Comet offers the most complete experiment tracking solution on the market. It’s brought significant value to our business.”
Olcay Cirit
Staff Research and Tech lead
“Comet enables us to speed up research cycles and reliably reproduce and collaborate on our modeling projects. It has become an indispensable part of our ML workflow.”
Victor Sanh
Machine Learning Scientist
“None of the other products have the simplicity, ease of use and feature set that Comet has.”
Ronny Huang
Research Scientist
“After discovering Comet, our deep learning team’s productivity went up. Comet is easy to set up and allows us to move research faster.”
Guru Rao
Head of AI
“We can seamlessly compare and share experiments, debug and stop underperforming models. Comet has improved our efficiency.”
Carol Anderson
Staff Data Scientist
[H2] Get started today, free.
You don’t need a credit card to sign up, and your Comet account comes with a generous free tier you can actually use—for as long as you like.
Try for Free
Get Demo

[H1] Let’s Talk
Whether you’re building LLM apps or scaling AI agents across your org, Opik is here to help.
Let us know what you’re working on and we’ll connect you with the right team.
TRUSTED BY THE MOST INNOVATIVE AI TEAMS
[IMG: AssemblyAI logo]
[IMG: Etsy logo]
[IMG: Uber logo]
[IMG: NatWest logo]
[IMG: Netflix logo]
[IMG: Shopify logo]
[H3] Contact Our Team
[H3] Thank you for contacting Comet!
You can schedule a meeting with our team using the link we’ve just sent to your email. We look forward to talking with you soon!
NEED PRODUCT SUPPORT?
[H2] We are here to support you and your team.
Send your product questions to us, and we will get in touch as soon as possible.
[H3] Documentation
Explore our detailed documentation for technical support.
See Docs
[H3] Slack Community
Join our slack community to receive tailored support in a fast manner.
Join Slack
[H3] Email Us
Reach out to us directly with technical questions so we can resolve quickly.
support@comet.com
[H3] Request Security Reports
Click here to request copies of our SOC 2 or ISO 27001 certifications.
Request Certs

ML Dataset Management for
[H1] Seamless Reproducibility
Comet Artifacts makes it easy to save and track datasets from training runs to production. It’s a dataset store built to handle even the most complex ML workflows and use cases.
Create Free Account
Book a Demo
[H2] Metadata Store for Collaboration Between Scientists and Engineers
Comet Artifacts provides a standardized process for better visibility and collaboration. Easily save, store, version, and link datasets to models in training and production.
[H3] Dataset Versioning
Easily save, store, version, and link datasets to models in training and production.
[IMG: screenshot of Comet Artifacts]
[H3] Dataset Metadata
Automatically track metadata on your datasets to support a standardized process for better visibility and collaboration.
[IMG: Artifacts Metadata Screenshot]
[H3] Dataset Lineage
Lineage allows your teammates to visualize how you created your model and used your datasets, in an interactive way, for easier reproducibility.
[IMG: Artifacts Screenshot Lineage]
[H2] Get started today.
You don’t need a credit card to sign up, and your Comet account comes with a generous free tier you can actually use—for as long as you like.
Create Free Account
Contact Sales