Amit Choubey
← Back to Posts

Fine-Tuning Mistral-7B for Sarcasm with LoRA and 4-Bit Quantization

August 5, 2025

Teaching an AI to Be Sarcastic: My Journey Fine-Tuning Mistral-7B

Last month, I fell down a rabbit hole that started with a simple question: "Can I teach a language model to be sarcastic with just a few hundred examples?" What followed was three weeks of late-night coding sessions, GPU memory errors, and another fine-tuned model in a long line of experiments—this one tuned for wit.

The result? sweatSmile/Mistral-7B-Instruct-v0.1-Sarcasm - a model that actually gets sarcasm (most of the time).

Why Sarcasm? Why Not Something Easier?

I'll be honest - I didn't choose sarcasm because it was easy. I chose it because it's fascinating how humans can completely flip the meaning of a sentence with just tone and context. Plus, I was getting tired of AI assistants that took everything literally.

Think about it: when someone says "Oh great, another meeting," they're not expressing joy. But for most LLMs, detecting that subtle negativity wrapped in positive words is genuinely challenging. I wanted to see if I could build something that would understand when I'm being sarcastic in my prompts.

The technical challenge was equally appealing. Sarcasm requires understanding context, cultural references, and the gap between literal meaning and intent. If I could get a 7B parameter model to handle this with minimal training data, it would prove something interesting about the effectiveness of targeted fine-tuning.

Building the Dataset: 700 Examples of Pure Snark

Creating a sarcasm dataset is weirdly personal. I started by collecting examples from Reddit threads, Twitter conversations, and honestly, transcribing myself during particularly sarcastic moments (my friends were very helpful here).

The dataset ended up being around 700 examples, each structured as conversation pairs:

Human: "I love working overtime on weekends"
Assistant: "Oh absolutely, because nothing says 'work-life balance' like spending your Saturday debugging someone else's code. Truly living the dream."

I made sure to include different types of sarcasm:

  • Situational sarcasm: Responding to obviously frustrating situations
  • Self-deprecating humor: The kind where you're sarcastic about your own mistakes
  • Social commentary: Gentle (and not-so-gentle) observations about modern life
  • Technical sarcasm: Because us developers have our own special brand of cynicism

The key was variety. I didn't want the model to just learn one pattern of sarcasm - I wanted it to understand the different flavors and contexts where sarcasm appears naturally.

Technical Deep Dive: Making Magic Happen with LoRA

Here's where things got interesting from a technical perspective. I used Low-Rank Adaptation (LoRA) because, frankly, it was my only option given the hardware constraints. But it turned out to be perfect for this use case.

The LoRA Configuration That Actually Worked

After multiple failed attempts (and a lot of OOM errors), I landed on these settings:

lora_config = LoraConfig(
    r=16,                    # Rank - sweet spot between capacity and efficiency
    lora_alpha=32,           # Scaling factor (2x the rank is a good rule)
    lora_dropout=0.05,       # Just enough to prevent overfitting
    target_modules=[
        "q_proj", "k_proj", "v_proj", "o_proj",  # Attention layers
        "gate_proj", "up_proj", "down_proj"       # MLP layers
    ],
    bias="none",
    task_type="CAUSAL_LM"
)

The r=16 was crucial. I initially tried r=64 thinking "bigger is better," but that led to overfitting on my small dataset. The model would memorize the training examples perfectly but couldn't generalize. Dropping to r=16 gave it just enough capacity to learn the sarcastic patterns without losing its general language abilities.

4-Bit Quantization: My GPU's Best Friend

The quantization setup was non-negotiable given my hardware:

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

This compressed the model from 14GB to about 4GB in memory. The tradeoff? Some precision loss, but honestly, for sarcasm detection, the slight degradation was completely acceptable. Plus, it meant I could actually run training without constantly hitting memory limits.

Training Configuration: Lessons from Many Failures

My final training setup came after numerous iterations:

  • Learning rate: 3e-4 (started at 5e-4, but the model was too aggressive in changing its personality)
  • Batch size: Effective batch size of 8 with gradient accumulation
  • Epochs: 10 (found that sweet spot where it learned sarcasm without forgetting how to be helpful)
  • Max sequence length: 64 tokens (limitation of my dataset, but sufficient for sarcastic one-liners)

The most important discovery was that less is more with personality fine-tuning. I initially planned for 20 epochs, but the model started becoming obnoxiously sarcastic about everything after epoch 12. At 10 epochs, it learned to be sarcastic when appropriate while maintaining its helpful nature for serious queries.

Real-World Testing: Does It Actually Work?

The proof, as they say, is in the pudding. After training, I spent hours testing the model with different scenarios:

Scenario 1: Monday Morning Blues

Me: "Another Monday, another opportunity to shine!"
Model: "Ah yes, nothing quite like that Monday morning energy where 'opportunity to shine' really means 'opportunity to question all my life choices while staring at an overflowing inbox.'"

Scenario 2: Code Review Comments

Me: "The code looks perfect, no issues at all"
Model: "Perfect code? In a code review? Well, that's a first. Let me guess - either you wrote it yourself or you're reviewing 'Hello World' for the thousandth time."

The model had learned to pick up on obviously false enthusiasm and respond with appropriate sarcasm. More importantly, when I asked genuine technical questions, it responded helpfully without the sarcastic tone.

The Unexpected Challenges

Challenge 1: The Overfitting Trap

With only 700 examples, overfitting was a constant threat. The model would start memorizing exact phrases from the training set. I solved this with aggressive dropout and early stopping, but it required constant vigilance during training.

Challenge 2: Context Window Limitations

My 64-token limit meant the model couldn't handle longer conversations or complex setups for sarcasm. This is definitely something I'd improve in version 2 with more compute resources.

Challenge 3: Cultural Bias in Sarcasm

Sarcasm is deeply cultural. My dataset was heavily influenced by American internet culture and developer humor. The model works great if you share that context, but it might miss sarcasm from other cultural backgrounds.

Challenge 4: Balancing Personality and Helpfulness

The trickiest part was keeping the model helpful for serious queries while making it appropriately sarcastic for obvious setups. This required careful curation of the training data and monitoring during training.

What I Learned About Fine-Tuning

This project taught me several things that you won't find in most tutorials:

  1. Small datasets can work, but they need to be perfect: Every example in my 700-sample dataset had to be high quality. There was no room for noise or inconsistency.

  2. Personality tuning is different from task tuning: Teaching a model to be sarcastic requires different considerations than teaching it to classify text or answer questions. The evaluation is more subjective, and the training dynamics are more delicate.

  3. Hardware constraints breed creativity: Being forced to use LoRA and quantization made me understand these techniques deeply. Sometimes limitations lead to better solutions.

  4. The merge back to base model is crucial: After training the LoRA adapter, merging it back into the base model made deployment much simpler. No need to manage separate adapter files in production.

Technical Takeaways for Your Own Projects

If you're thinking about doing something similar, here are the key insights:

  • Start with a clear personality goal: "Sarcastic but helpful" was my north star. Without this clarity, you'll end up with an inconsistent model.

  • Invest time in data quality over quantity: 700 high-quality examples beat 5000 mediocre ones for personality tuning.

  • Monitor training closely: With small datasets, the difference between "perfectly tuned" and "completely broken" can be just a few epochs.

  • Test extensively with edge cases: Sarcasm models can easily become offensive or inappropriate. Test with sensitive topics and boundary cases.

What's Next?

I'm already working on version 2 with several improvements:

  • Longer context window: 256 tokens to handle more complex conversational setups
  • Larger dataset: Aiming for 2000+ examples with more diverse cultural contexts
  • Better evaluation: Building automated tests for sarcasm detection and appropriateness

I'm also considering fine-tuning for other personality traits. Maybe a model that's overly optimistic? Or one that speaks like a film noir detective? The possibilities are endless.

Try It Yourself

The model is live on Hugging Face: sweatSmile/Mistral-7B-Instruct-v0.1-Sarcasm

Feel free to experiment with it, and let me know what you discover. I'm particularly interested in edge cases where it fails or succeeds unexpectedly.

Final Thoughts

This project reminded me why I love working with AI. It's not just about the technical challenge (though that's fun too) - it's about exploring the boundaries between human and machine communication. Teaching a computer to understand sarcasm feels like a small step toward more natural human-AI interaction.

Plus, now I have an AI that can match my level of snark. What more could a developer ask for?

Want to build your own personality-tuned model? I'm always up for chatting about LoRA, quantization, or the finer points of teaching machines to be sarcastic. Hit me up!