updated
Browse files- FinAraT5.png +3 -0
- README.md +53 -12
FinAraT5.png
ADDED
|
Git LFS Details
|
README.md
CHANGED
|
@@ -17,17 +17,25 @@ model-index:
|
|
| 17 |
results: []
|
| 18 |
---
|
| 19 |
|
| 20 |
-
#
|
| 21 |
|
| 22 |
-
This model is a fine-tuned version of **FinAraT5<sub>MSA</sub>** on Alarabya-news-summarisation dataset.
|
| 23 |
|
| 24 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
|
|
|
|
| 26 |
|
| 27 |
-
|
|
|
|
| 28 |
|
| 29 |
-
|
| 30 |
|
|
|
|
|
|
|
| 31 |
|
| 32 |
### Training hyperparameters
|
| 33 |
|
|
@@ -40,9 +48,44 @@ The following hyperparameters were used during training:
|
|
| 40 |
- lr_scheduler_type: linear
|
| 41 |
- num_epochs: 22.0
|
| 42 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
|
| 44 |
|
| 45 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
Below is an example for fine-tuning any text-to-text model for News Title Generation on any dataset
|
| 47 |
|
| 48 |
``` bash
|
|
@@ -65,8 +108,8 @@ Below is an example for fine-tuning any text-to-text model for News Title Gener
|
|
| 65 |
--push_to_hub_token "<HF_ID>" \
|
| 66 |
--report_to "wandb" \
|
| 67 |
--run_name "fine tuning a model on financial arabic news summarization dataset" \
|
| 68 |
-
```
|
| 69 |
|
|
|
|
| 70 |
|
| 71 |
### Framework versions
|
| 72 |
|
|
@@ -75,9 +118,7 @@ Below is an example for fine-tuning any text-to-text model for News Title Gener
|
|
| 75 |
- Datasets 2.5.1
|
| 76 |
- Tokenizers 0.13.0
|
| 77 |
|
|
|
|
|
|
|
| 78 |
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
## Acknowledgments
|
| 82 |
-
We gratefully acknowledge the [Google TensorFlow Research Cloud (TFRC)](https://www.tensorflow.org/tfrc) program for the free TPU V3.8 access and we thank the google cloud team for the free GCP credits.
|
| 83 |
-
|
|
|
|
| 17 |
results: []
|
| 18 |
---
|
| 19 |
|
| 20 |
+
# FinAraT5 – A T5-based Arabic Financial Text Generation Model
|
| 21 |
|
| 22 |
+
**FinAraT5** is the first Arabic *financial domain* T5-based text-to-text model. This model is a fine-tuned version of **FinAraT5<sub>MSA</sub>** on Alarabya-news-summarisation dataset. The model is based on [AraT5](https://huggingface.co/aubmindlab/araT5-base) and trained using domain-specific financial Arabic corpora.
|
| 23 |
|
| 24 |
+
> 📘 [Official Paper (LDK 2023)](https://aclanthology.org/2023.ldk-1.25)
|
| 25 |
+
> 📘 Authors: [Nadhem Zmandar](https://www.linkedin.com/in/nadhem-zmandar/), [Mo El-Haj](https://elhaj.uk/), and [Paul Rayson](https://www.lancaster.ac.uk/lira/people/paul-rayson)
|
| 26 |
+
---
|
| 27 |
+
|
| 28 |
+
## 🔧 Model Use Case
|
| 29 |
|
| 30 |
+
This model is designed for:
|
| 31 |
|
| 32 |
+
Generating short, informative headlines for Arabic financial news articles
|
| 33 |
+
Summarising long financial texts into concise titles or summary statements
|
| 34 |
|
| 35 |
+
It can assist news agencies, financial analysts, and media platforms in streamlining content production.
|
| 36 |
|
| 37 |
+
⚠️ Note: The model was fine-tuned on data collected from a single source (Al Arabiya), which may limit generalisability to other domains or styles.
|
| 38 |
+
---
|
| 39 |
|
| 40 |
### Training hyperparameters
|
| 41 |
|
|
|
|
| 48 |
- lr_scheduler_type: linear
|
| 49 |
- num_epochs: 22.0
|
| 50 |
|
| 51 |
+
## 💡 Example Usage
|
| 52 |
+
|
| 53 |
+
```python
|
| 54 |
+
from transformers import T5Tokenizer, T5ForConditionalGeneration
|
| 55 |
+
import torch
|
| 56 |
+
|
| 57 |
+
model = T5ForConditionalGeneration.from_pretrained("drelhaj/FinAraT5")
|
| 58 |
+
tokenizer = T5Tokenizer(vocab_file="spiece.model") # If required
|
| 59 |
+
|
| 60 |
+
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
| 61 |
+
model.to(device)
|
| 62 |
+
model.eval()
|
| 63 |
+
|
| 64 |
+
input_text = "أعلنت الشركة عن ارتفاع أرباحها بنسبة ١٥٪ في الربع الثاني من العام نتيجة لزيادة المبيعات في السوق الخليجية"
|
| 65 |
+
inputs = tokenizer(input_text, return_tensors="pt")
|
| 66 |
+
input_ids = inputs["input_ids"].to(device)
|
| 67 |
+
attention_mask = inputs["attention_mask"].to(device)
|
| 68 |
+
|
| 69 |
+
with torch.no_grad():
|
| 70 |
+
outputs = model.generate(input_ids=input_ids, attention_mask=attention_mask, max_length=30)
|
| 71 |
+
|
| 72 |
+
summary = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
| 73 |
+
print(summary)
|
| 74 |
+
```
|
| 75 |
|
| 76 |
|
| 77 |
+
💡 If using this model locally, ensure that spiece.model is included in the model directory for proper tokenisation.
|
| 78 |
+
|
| 79 |
+
## 📝 Example Output
|
| 80 |
+
- input_text: 'صعدت أسعار الذهب على نحو طفيف اليوم الاثنين، حيث أدى ارتفاع التضخم في الولايات المتحدة إلى تعزيز جاذبيته كملاذ آمن، في حين يترقب المستثمرون اجتماع مجلس الاحتياطي الاتحادي لمعرفة مدى السرعة التي يعتزم بها إلغاء برنامج شراء السندات., وارتفع الذهب في المعاملات الفورية 0.2% إلى 1785.20 دولار للأونصة، وزادت العقود الأميركية الآجلة للذهب 0.1% إلى 1785.70 دولار., ومن المرجح أن يعلن مجلس الاحتياطي الاتحادي (البنك المركزي الأميركي) عن خفض أسرع في مشتريات السندات لكن المخاوف الأكثر وضوحا بشأن التضخم يمكن أن تزعج الأسواق., ورغم أن الذهب يعتبر أداة للتحوط من التضخم، فإن خفض التحفيز ورفع أسعار الفائدة عادة ما يؤديان إلى دفع عوائد السندات الحكومية للصعود، مما يرفع تكلفة الفرصة البديلة لحيازة المعدن الأصفر الذي لا يدر عائدا., وتتجه الأنظار الآن إلى اجتماع مجلس الاحتياطي المقرر في 14-15 ديسمبر/ كانون الأول., وارتفعت الفضة في المعاملات الفورية 0.3% إلى 22.22 دولار للأونصة., وزاد البلاتين 0.5% إلى 946.74 دولار، وارتفع البلاديوم 0.5% إلى 1769.61 دولار.'
|
| 81 |
+
|
| 82 |
+
|
| 83 |
+
Output:
|
| 84 |
+
الذهب يصعد مع ارتفاع التضخم في أميركا
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
|
| 88 |
+
## 🏗️ Fine-tuning Example
|
| 89 |
Below is an example for fine-tuning any text-to-text model for News Title Generation on any dataset
|
| 90 |
|
| 91 |
``` bash
|
|
|
|
| 108 |
--push_to_hub_token "<HF_ID>" \
|
| 109 |
--report_to "wandb" \
|
| 110 |
--run_name "fine tuning a model on financial arabic news summarization dataset" \
|
|
|
|
| 111 |
|
| 112 |
+
```
|
| 113 |
|
| 114 |
### Framework versions
|
| 115 |
|
|
|
|
| 118 |
- Datasets 2.5.1
|
| 119 |
- Tokenizers 0.13.0
|
| 120 |
|
| 121 |
+
## 🙏 Acknowledgements
|
| 122 |
+
Many thanks to Dr Nadhem Zmandar (AI Research Engineer) for his great effort into building this model. Please get in touch with Nadhem on: [LinkedIn:](https://www.linkedin.com/in/nadhem-zmandar/). Nadhem did this work as part of his PhD thesis titled [Multilingual Financial Text Summarisation](https://eprints.lancs.ac.uk/id/eprint/225264/). For any other questions please contact Dr Mo El-Haj [https://elhaj.uk](https://elhaj.uk).
|
| 123 |
|
| 124 |
+
We gratefully acknowledge the [Google TensorFlow Research Cloud (TFRC)](https://www.tensorflow.org/tfrc) program for the free TPU V3.8 access and we thank the google cloud team for the free GCP credits.
|
|
|
|
|
|
|
|
|
|
|
|