primel/aiba-bert-bilstm

AIBA BERT-BiLSTM Multi-Task Model

Multi-task BERT-BiLSTM model for banking transaction processing in multiple languages.

🎯 Tasks

1. Named Entity Recognition (NER) - Extract transaction entities (receiver name, INN, amount, etc.)
2. Intent Classification - Classify transaction intent (create, list, partial)
3. Language Detection - Detect input language (Uzbek Latin/Cyrillic, Russian, English, Mixed)

📊 Performance

Task	Metric	Score
NER	F1	0.9941
Intent	F1	0.9611
Language	F1	0.9146
Average	F1	0.9566

🚀 Quick Start

Installation

pip install transformers torch safetensors huggingface-hub

Basic Usage

from nn_model import load_model_and_tokenizer
import torch

# Load model
model, tokenizer, config = load_model_and_tokenizer("YOUR_USERNAME/aiba-bert-bilstm")

# Example text
text = "Qabul qiluvchi Omad Biznes MCHJ STIR 123456789 summa 500000 UZS"

# Tokenize
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=192)

# Predict
with torch.no_grad():
    outputs = model(**inputs)

# Get results
ner_logits = outputs['ner_logits']
intent_logits = outputs['intent_logits']
lang_logits = outputs['lang_logits']

Using the Inference Script

from inference import predict

result = predict("Получатель ООО Прогресс ИНН 987654321 сумма 1000000 руб")

print(f"Intent: {result['intent']}")
print(f"Language: {result['language']}")
print(f"Entities: {result['entities']}")

📦 Model Architecture

• Base: google-bert/bert-base-multilingual-uncased
• NER Head: BiLSTM (256 hidden units) + Linear classifier
• Pooling: Attention-based pooling for sequence tasks
• Intent Head: 3-layer MLP with GELU activation
• Language Head: 2-layer MLP with GELU activation

🏷️ Labels

NER Entities (16)

agreement_number_indicator, amount, date, description, indicator_amount, indicator_date, indicator_description, indicator_receiver_hr, indicator_receiver_inn, indicator_receiver_name, indicator_status, mfo, receiver_hr, receiver_inn, receiver_name, status

Intents (3)

create_transaction, list_transaction, partial_entities

Languages (5)

en, mixed, ru, uz_cyrl, uz_latn

📂 Repository Structure

.
├── config.json              # Model configuration
├── pytorch_model.bin        # Model weights
├── model.safetensors       # Model weights (safetensors format)
├── tokenizer_config.json   # Tokenizer configuration
├── nn_model.py             # Model architecture code
├── inference.py            # Example inference script
├── requirements.txt        # Python dependencies
└── README.md              # This file

🔧 Training Details

• Epochs: 3
• Learning Rate: 8e-06
• Batch Size: 48
• Max Sequence Length: 192
• Optimizer: AdamW
• Loss Weights: NER=0.5, Intent=0.35, Lang=0.15

💡 Use Cases

• Banking Chatbots: Extract transaction details from user messages
• Payment Processing: Automatically parse payment instructions
• Transaction Routing: Classify transaction type and route appropriately
• Multilingual Support: Handle inputs in Uzbek, Russian, and English

⚠️ Limitations

• Maximum sequence length: 192 tokens
• Trained on banking/financial domain data
• Best performance on transaction-related text

📄 Citation

@misc{aiba-bert-bilstm,
  title = {AIBA BERT-BiLSTM Multi-Task Model},
  author = {YOUR_USERNAME},
  year = {2024},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/YOUR_USERNAME/aiba-bert-bilstm}}
}

📧 Contact

For questions or issues, please open an issue on the repository.

📜 License

Apache 2.0