Add model card with exact and within-1 confusion matrices and per-class metrics

README.md

@@ -1,9 +1,9 @@
 
-# Fine-Tuned Gemma-7B CEFR Model
+# Fine-Tuned Mistral-7B CEFR Model
 
-This is a fine-tuned version of `unsloth/gemma-7b-bnb-4bit` for CEFR-level sentence generation, evaluated with a fine-tuned classifier from `Mr-FineTuner/Skripsi_validator_best_model`.
+This is a fine-tuned version of `unsloth/mistral-7b-bnb-4bit` for CEFR-level sentence generation, evaluated with a fine-tuned classifier from `Mr-FineTuner/Skripsi_validator_best_model`.
 
-- **Base Model**: unsloth/gemma-7b-bnb-4bit
+- **Base Model**: unsloth/mistral-7b-bnb-4bit
 - **Fine-Tuning**: LoRA with balanced dataset
 - **Training Details**:
   - Dataset: CEFR-level sentences (balanced)
@@ -12,21 +12,21 @@ This is a fine-tuned version of `unsloth/gemma-7b-bnb-4bit` for CEFR-level sente
   - Optimizer: adamw_8bit
   - Early Stopping: Patience=3, threshold=0.01
 - **Evaluation Metrics (Exact Matches)**:
-  - CEFR Classifier Accuracy: 0.167
-  - Precision (Macro): 0.033
-  - Recall (Macro): 0.167
-  - F1-Score (Macro): 0.056
+  - CEFR Classifier Accuracy: 0.500
+  - Precision (Macro): 0.333
+  - Recall (Macro): 0.500
+  - F1-Score (Macro): 0.389
 - **Evaluation Metrics (Within ±1 Level)**:
-  - CEFR Classifier Accuracy: 0.667
-  - Precision (Macro): 0.556
-  - Recall (Macro): 0.667
-  - F1-Score (Macro): 0.583
+  - CEFR Classifier Accuracy: 0.833
+  - Precision (Macro): 0.750
+  - Recall (Macro): 0.833
+  - F1-Score (Macro): 0.778
 - **Other Metrics**:
-  - Perplexity: 2.952
+  - Perplexity: 2.404
   - Diversity (Unique Sentences): 0.100
-  - Inference Time (ms): 6696.712
+  - Inference Time (ms): 6267.856
   - Model Size (GB): 4.2
-  - Robustness (F1): 0.053
+  - Robustness (F1): 0.369
 - **Confusion Matrix (Exact Matches)**:
   - CSV: [confusion_matrix_exact.csv](confusion_matrix_exact.csv)
   - Image: [confusion_matrix_exact.png](confusion_matrix_exact.png)
@@ -35,24 +35,24 @@ This is a fine-tuned version of `unsloth/gemma-7b-bnb-4bit` for CEFR-level sente
   - Image: [confusion_matrix_within1.png](confusion_matrix_within1.png)
 - **Per-Class Confusion Metrics (Exact Matches)**:
   - A1: TP=0, FP=0, FN=10, TN=50
-  - A2: TP=10, FP=40, FN=0, TN=10
-  - B1: TP=0, FP=0, FN=10, TN=50
-  - B2: TP=0, FP=0, FN=10, TN=50
-  - C1: TP=0, FP=10, FN=10, TN=40
+  - A2: TP=0, FP=10, FN=10, TN=40
+  - B1: TP=10, FP=10, FN=0, TN=40
+  - B2: TP=10, FP=10, FN=0, TN=40
+  - C1: TP=10, FP=0, FN=0, TN=50
   - C2: TP=0, FP=0, FN=10, TN=50
 - **Per-Class Confusion Metrics (Within ±1 Level)**:
   - A1: TP=10, FP=0, FN=0, TN=50
-  - A2: TP=10, FP=20, FN=0, TN=30
+  - A2: TP=10, FP=0, FN=0, TN=50
   - B1: TP=10, FP=0, FN=0, TN=50
-  - B2: TP=0, FP=0, FN=10, TN=50
-  - C1: TP=0, FP=0, FN=10, TN=50
-  - C2: TP=10, FP=0, FN=0, TN=50
+  - B2: TP=10, FP=10, FN=0, TN=40
+  - C1: TP=10, FP=0, FN=0, TN=50
+  - C2: TP=0, FP=0, FN=10, TN=50
 - **Usage**:
   ```python
   from transformers import AutoModelForCausalLM, AutoTokenizer
 
-  model = AutoModelForCausalLM.from_pretrained("Mr-FineTuner/Test_02_gemma_trainPercen_myValidator")
-  tokenizer = AutoTokenizer.from_pretrained("Mr-FineTuner/Test_02_gemma_trainPercen_myValidator")
+  model = AutoModelForCausalLM.from_pretrained("Mr-FineTuner/With_synthetic_Dataset_mistral-1epoch")
+  tokenizer = AutoTokenizer.from_pretrained("Mr-FineTuner/With_synthetic_Dataset_mistral-1epoch")
 
   # Example inference
   prompt = "<|user|>Generate a CEFR B1 level sentence.<|end|>"