Mr-FineTuner/With_synthetic_Dataset_llama-001epoch

Fine-Tuned LLaMA-3-8B CEFR Model

This is a fine-tuned version of unsloth/llama-3-8b-instruct-bnb-4bit for CEFR-level sentence generation, evaluated with a fine-tuned classifier from Mr-FineTuner/Skripsi_validator_best_model.

• Base Model: unsloth/llama-3-8b-instruct-bnb-4bit
• Fine-Tuning: LoRA with balanced dataset
• Training Details:
  • Dataset: CEFR-level sentences
  • LoRA Parameters: r=32, lora_alpha=32, lora_dropout=0.5
  • Training Args: learning_rate=1e-5, batch_size=8, epochs=0.01, cosine scheduler
  • Optimizer: adamw_8bit
  • Early Stopping: Patience=2, threshold=0.01
• Evaluation Metrics (Exact Matches):
  • CEFR Classifier Accuracy: 0.283
  • Precision (Macro): 0.267
  • Recall (Macro): 0.283
  • F1-Score (Macro): 0.267
• Evaluation Metrics (Within ±1 Level):
  • CEFR Classifier Accuracy: 0.850
  • Precision (Macro): 0.878
  • Recall (Macro): 0.850
  • F1-Score (Macro): 0.838
• Other Metrics:
  • Perplexity: 22.487
  • Diversity (Unique Sentences): 0.983
  • Inference Time (ms): 5981.186
  • Model Size (GB): 8.0 # Updated to reflect PyTorch format
  • Robustness (F1): 0.253
• Confusion Matrix (Exact Matches):
  • CSV: confusion_matrix_exact.csv
  • Image: confusion_matrix_exact.png
• Confusion Matrix (Within ±1 Level):
  • CSV: confusion_matrix_within1.csv
  • Image: confusion_matrix_within1.png
• Per-Class Confusion Metrics (Exact Matches):
  • A1: TP=0, FP=0, FN=10, TN=50
  • A2: TP=2, FP=5, FN=8, TN=45
  • B1: TP=3, FP=17, FN=7, TN=33
  • B2: TP=4, FP=11, FN=6, TN=39
  • C1: TP=4, FP=6, FN=6, TN=44
  • C2: TP=4, FP=4, FN=6, TN=46
• Per-Class Confusion Metrics (Within ±1 Level):
  • A1: TP=4, FP=0, FN=6, TN=50
  • A2: TP=9, FP=1, FN=1, TN=49
  • B1: TP=9, FP=5, FN=1, TN=45
  • B2: TP=9, FP=2, FN=1, TN=48
  • C1: TP=10, FP=0, FN=0, TN=50
  • C2: TP=10, FP=1, FN=0, TN=49
• Usage:

  from transformers import AutoModelForCausalLM, AutoTokenizer
  
  model = AutoModelForCausalLM.from_pretrained("Mr-FineTuner/With_synthetic_Dataset_llama-001epoch")
  tokenizer = AutoTokenizer.from_pretrained("Mr-FineTuner/With_synthetic_Dataset_llama-001epoch")
  
  # Example inference
  prompt = "<|user|>Generate a CEFR B1 level sentence.<|end|>"
  inputs = tokenizer(prompt, return_tensors="pt")
  outputs = model.generate(**inputs, max_length=50)
  print(tokenizer.decode(outputs[0], skip_special_tokens=True))
  Uploaded using huggingface_hub.