Add model card with exact and within-1 confusion matrices and per-class metrics

README.md

@@ -0,0 +1,64 @@
+
+# Fine-Tuned Mistral-7B CEFR Model
+
+This is a fine-tuned version of `unsloth/mistral-7b-instruct-v0.3-bnb-4bit` for CEFR-level sentence generation, evaluated with a fine-tuned classifier from `Mr-FineTuner/Skripsi_validator_best_model`.
+
+- **Base Model**: unsloth/mistral-7b-instruct-v0.3-bnb-4bit
+- **Fine-Tuning**: LoRA with SMOTE-balanced dataset
+- **Training Details**:
+  - Dataset: CEFR-level sentences with SMOTE and undersampling for balance
+  - LoRA Parameters: r=32, lora_alpha=32, lora_dropout=0.5
+  - Training Args: learning_rate=2e-5, batch_size=8, epochs=0.01, cosine scheduler
+  - Optimizer: adamw_8bit
+  - Early Stopping: Patience=3, threshold=0.01
+- **Evaluation Metrics (Exact Matches)**:
+  - CEFR Classifier Accuracy: 0.000
+  - Precision (Macro): 0.000
+  - Recall (Macro): 0.000
+  - F1-Score (Macro): 0.000
+- **Evaluation Metrics (Within ±1 Level)**:
+  - CEFR Classifier Accuracy: 0.500
+  - Precision (Macro): 0.333
+  - Recall (Macro): 0.500
+  - F1-Score (Macro): 0.389
+- **Other Metrics**:
+  - Perplexity: 6.089
+  - Diversity (Unique Sentences): 0.100
+  - Inference Time (ms): 5150.096
+  - Model Size (GB): 4.1
+  - Robustness (F1): 0.000
+- **Confusion Matrix (Exact Matches)**:
+  - CSV: [confusion_matrix_exact.csv](confusion_matrix_exact.csv)
+  - Image: [confusion_matrix_exact.png](confusion_matrix_exact.png)
+- **Confusion Matrix (Within ±1 Level)**:
+  - CSV: [confusion_matrix_within1.csv](confusion_matrix_within1.csv)
+  - Image: [confusion_matrix_within1.png](confusion_matrix_within1.png)
+- **Per-Class Confusion Metrics (Exact Matches)**:
+  - A1: TP=0, FP=0, FN=10, TN=50
+  - A2: TP=0, FP=0, FN=10, TN=50
+  - B1: TP=0, FP=10, FN=10, TN=40
+  - B2: TP=0, FP=0, FN=10, TN=50
+  - C1: TP=0, FP=30, FN=10, TN=20
+  - C2: TP=0, FP=20, FN=10, TN=30
+- **Per-Class Confusion Metrics (Within ±1 Level)**:
+  - A1: TP=0, FP=0, FN=10, TN=50
+  - A2: TP=0, FP=0, FN=10, TN=50
+  - B1: TP=0, FP=10, FN=10, TN=40
+  - B2: TP=10, FP=0, FN=0, TN=50
+  - C1: TP=10, FP=10, FN=0, TN=40
+  - C2: TP=10, FP=10, FN=0, TN=40
+- **Usage**:
+  ```python
+  from transformers import AutoModelForCausalLM, AutoTokenizer
+
+  model = AutoModelForCausalLM.from_pretrained("Mr-FineTuner/Test_01_withNewEval_andWithin-1_mistral_skripsi_classifier")
+  tokenizer = AutoTokenizer.from_pretrained("Mr-FineTuner/Test_01_withNewEval_andWithin-1_mistral_skripsi_classifier")
+
+  # Example inference
+  prompt = "<|user|>Generate a CEFR B1 level sentence.<|end|>"
+  inputs = tokenizer(prompt, return_tensors="pt")
+  outputs = model.generate(**inputs, max_length=50)
+  print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+  ```
+
+Uploaded using `huggingface_hub`.