Classifier Export & Training

Learn how to export trained classifiers, classify new documents without LLM costs, and train classifiers from your own labeled data.

Overview

Delve’s classifier workflow enables cost-effective production use:

Export: Save a trained classifier after any Delve run
Classify: Label new documents using only embeddings (no LLM)
Retrain: Improve classifiers with corrected/curated data

Exporting a Classifier

After running Delve, save the classifier for later use:

from delve import Delve

delve = Delve(sample_size=100)
result = delve.run_sync("data.csv", text_column="text")

# Save the trained classifier
result.save_classifier("classifier.joblib")

What’s Saved

The .joblib bundle contains:

Trained RandomForest model
Category index mappings
Embedding model name (for consistency)
Full taxonomy with descriptions
Training metrics

A classifier is only available when sample_size < total documents. If all documents were labeled by the LLM (no classifier was trained), save_classifier() will raise an error.

Classifying New Documents

Load a saved classifier and classify documents with no LLM cost:

from delve import Delve

predictions = Delve.classify(
    "new_data.csv",
    classifier_path="classifier.joblib",
    text_column="text",
)

# Access results
for doc in predictions.documents:
    print(f"{doc.id}: {doc.category} (confidence: {doc.confidence:.2%})")

Cost Comparison

Method	LLM Cost	Embedding Cost
Full Delve run	High	Medium
Saved classifier	None	Low

Accessing Results

# As list of Doc objects
for doc in predictions.documents:
    print(f"{doc.id}: {doc.category} (confidence: {doc.confidence:.2f})")

# As DataFrame
df = predictions.to_dataframe()
print(df.head())

# Export to file
predictions.export("./output", formats=["csv", "json"])

API Options

predictions = Delve.classify(
    data="new_data.csv",           # CSV, JSON, DataFrame, or List[Doc]
    classifier_path="classifier.joblib",
    text_column="text",            # Required for CSV/DataFrame
    id_column="doc_id",            # Optional: column for document IDs
    include_confidence=True,       # Include confidence scores (default: True)
    verbosity=Verbosity.NORMAL,    # Output verbosity
)

Training from Labeled Data

Train a classifier directly from your labeled dataset:

from delve import Delve

result = Delve.train_from_labeled(
    "labeled_data.csv",
    text_column="text",
    label_column="category",
)

print(f"Test Accuracy: {result.metrics['test_accuracy']:.2%}")
print(f"Test F1: {result.metrics['test_f1']:.2%}")

# Save for production use
result.save_classifier("production_classifier.joblib")

When to Use This

You have manually labeled data
You’ve corrected Delve’s output
You want to combine multiple labeled datasets
You’re creating a production classifier from curated examples

With Explicit Taxonomy

Provide a taxonomy for consistent category descriptions:

taxonomy = [
    {"id": "1", "name": "Bug", "description": "Software bugs and defects"},
    {"id": "2", "name": "Feature", "description": "Feature requests and enhancements"},
    {"id": "3", "name": "Question", "description": "General questions"},
]

result = Delve.train_from_labeled(
    "labeled_data.csv",
    text_column="text",
    label_column="category",
    taxonomy=taxonomy,
)

If no taxonomy is provided, one is inferred from the unique labels in your data.

Checking Quality

print(f"Training samples: {result.training_docs_count}")
print(f"Test samples: {result.validation_docs_count}")
print(f"Test Accuracy: {result.metrics['test_accuracy']:.2%}")
print(f"Test F1: {result.metrics['test_f1']:.2%}")

# Per-class performance
for cat, f1 in result.metrics['per_class_f1'].items():
    print(f"  {cat}: {f1:.2f}")

API Options

result = Delve.train_from_labeled(
    data="labeled_data.csv",       # CSV, JSON, or DataFrame
    text_column="text",            # Column with document text
    label_column="category",       # Column with labels
    id_column="doc_id",            # Optional: column for document IDs
    taxonomy="taxonomy.json",      # Optional: explicit taxonomy
    embedding_model="text-embedding-3-large",  # Embedding model
    test_size=0.2,                 # Validation split (default: 20%)
    verbosity=Verbosity.NORMAL,    # Output verbosity
)

Human-in-the-Loop Workflow

Combine Delve’s automation with human expertise:

Step 1: Initial Run

from delve import Delve, Verbosity

delve = Delve(sample_size=200, verbosity=Verbosity.VERBOSE)
result = delve.run_sync("training_data.csv", text_column="content")

# Export for human review
await result.export()  # Creates labeled_documents.csv

Step 2: Human Review

Review labeled_documents.csv and correct mislabeled documents. Focus on:

Low-confidence predictions
“Other” category documents
Edge cases between similar categories

Step 3: Retrain from Corrected Data

# Train improved classifier from corrected labels
result = Delve.train_from_labeled(
    "corrected_labels.csv",
    text_column="content",
    label_column="category",
    taxonomy="taxonomy.json",  # Use original taxonomy
)

print(f"Improved Test F1: {result.metrics['test_f1']:.2%}")
result.save_classifier("production_classifier.joblib")

Step 4: Production Classification

# Classify new documents with no LLM cost
predictions = Delve.classify(
    "new_documents.csv",
    classifier_path="production_classifier.joblib",
    text_column="content",
)

# Export results
df = predictions.to_dataframe()
df.to_csv("classified_documents.csv", index=False)

Focus human review on low-confidence predictions and “Other” categories - these benefit most from correction.

Async API

Both methods have async versions for use in async applications:

import asyncio
from delve import Delve

async def main():
    # Classify async
    predictions = await Delve.classify_async(
        "new_data.csv",
        classifier_path="classifier.joblib",
        text_column="text",
    )

    # Train async
    result = await Delve.train_from_labeled_async(
        "labeled_data.csv",
        text_column="text",
        label_column="category",
    )

asyncio.run(main())

Result Classes

ClassificationResult

Returned by Delve.classify():

@dataclass
class ClassificationResult:
    documents: List[Doc]           # Classified docs with category + confidence
    classifier_info: Dict[str, Any]  # Metadata about classifier used

    def to_dataframe(self) -> pd.DataFrame
    def to_dict(self) -> Dict[str, Any]
    def export(self, output_dir, formats=["csv"]) -> Dict[str, Path]

TrainingResult

Returned by Delve.train_from_labeled():

@dataclass
class TrainingResult:
    model: RandomForestClassifier
    index_to_category: Dict[int, str]
    taxonomy: List[TaxonomyCategory]
    metrics: Dict[str, Any]          # train/test accuracy, F1, per_class_f1
    training_docs_count: int
    validation_docs_count: int
    embedding_model: str
    created_at: str

    def save_classifier(self, path) -> Path
    def to_dict(self) -> Dict[str, Any]

Getting Started

Advanced Topics

CLI Usage

SDK Usage

Examples

Classifier Export & Training

Overview

Exporting a Classifier

What’s Saved

Classifying New Documents

Cost Comparison

Accessing Results

API Options

Training from Labeled Data

When to Use This

With Explicit Taxonomy

Checking Quality

API Options

Human-in-the-Loop Workflow

Step 1: Initial Run

Step 2: Human Review

Step 3: Retrain from Corrected Data

Step 4: Production Classification

Async API

Result Classes

ClassificationResult

TrainingResult

Next Steps

Class Imbalance

Configuration Guide

​Overview

​Exporting a Classifier

​What’s Saved

​Classifying New Documents

​Cost Comparison

​Accessing Results

​API Options

​Training from Labeled Data

​When to Use This

​With Explicit Taxonomy

​Checking Quality

​API Options

​Human-in-the-Loop Workflow

​Step 1: Initial Run

​Step 2: Human Review

​Step 3: Retrain from Corrected Data

​Step 4: Production Classification

​Async API

​Result Classes

​ClassificationResult

​TrainingResult

​Next Steps

Class Imbalance

Configuration Guide

Overview

Exporting a Classifier

What’s Saved

Classifying New Documents

Cost Comparison

Accessing Results

API Options

Training from Labeled Data

When to Use This

With Explicit Taxonomy

Checking Quality

API Options

Human-in-the-Loop Workflow

Step 1: Initial Run

Step 2: Human Review

Step 3: Retrain from Corrected Data

Step 4: Production Classification

Async API

Result Classes

ClassificationResult

TrainingResult

Next Steps