Doğal dil işlemede ileri seviye metin sınıflandırma: Transformer'ın rolü
Küçük Resim Yok
Tarih
2025
Yazarlar
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
İnönü Üniversitesi
Erişim Hakkı
info:eu-repo/semantics/openAccess
Özet
Bu tez çalışması, doğal dil işleme (Natural Language Processing, NLP) alanında metin sınıflandırma problemlerine yönelik geliştirilmiş geleneksel yöntemler (Naive Bayes ve Destek Vektör Makineleri) ile son yıllarda büyük başarı gösteren Transformer tabanlı modellerin (BERT, RoBERTa, GPT, T5, DistilBERT) karşılaştırmalı analizini kapsamaktadır. Çalışmanın temel amacı, farklı veri setlerinde bu yöntemlerin sınıflandırma performanslarını ölçmek ve elde edilen sonuçları doğruluk, kayıp oranları ve genel model başarısı açısından detaylı şekilde irdelemektir. Tez kapsamında AG News, IMDb ve SST-2 veri setleri kullanılmış; metin verileri uygun ön işleme süreçlerinden geçirilmiş ve hem geleneksel hem de Transformer tabanlı modellere entegre edilmiştir. Naive Bayes ve SVM gibi yöntemler, metin özelliklerini temel istatistiksel özellikler üzerinden değerlendirirken, Transformer tabanlı modeller ise dikkat mekanizması (attention mechanism) ve büyük ölçekli önceden eğitilmiş dil modelleri sayesinde bağlamsal bilgi zenginliğinden yararlanmıştır. Uygulama ve deneyler aşamasında her bir model veri setleri üzerinde eğitilmiş ve test edilmiştir. Sonuçlar, Transformer modellerinin genel olarak geleneksel yöntemlere kıyasla daha yüksek doğruluk oranlarına ulaştığını, ancak hesaplama maliyetleri ve eğitim süresi açısından geleneksel yöntemlerin hala avantajlı olduğunu ortaya koymuştur. Ayrıca, farklı modellerin sınıflandırma başarısı veri setine ve problem türüne göre değişiklik göstermektedir; örneğin, RoBERTa modeli AG News ve SST-2 veri setlerinde en yüksek doğruluk oranını sağlarken, IMDb veri setinde BERT modeli öne çıkmıştır. Bu tezde elde edilen bulgular, gelecekteki NLP uygulamaları için model seçiminde yol gösterici nitelikte olup, hem akademik hem de endüstriyel uygulamalara katkı sağlamayı hedeflemektedir. Sonuçlar, ayrıca Transformer mimarisinin güçlü yanlarının yanı sıra sınırlılıklarına da dikkat çekmekte ve daha verimli ve etkin yöntemler geliştirmek için bir temel sunmaktadır.
This thesis covers the comparative analysis of traditional methods (Naive Bayes and Support Vector Machines) developed for text classification problems in the field of Natural Language Processing (NLP) and Transformer-based models (BERT, RoBERTa, GPT, T5, DistilBERT) that have shown great success in recent years. The main purpose of the study is to measure the classification performances of these methods on different data sets and to examine the obtained results in detail in terms of accuracy, loss rates and general model success. Within the scope of the thesis, AG News, IMDb and SST-2 datasets were used; text data was subjected to appropriate pre-processing processes and integrated into both traditional and Transformer-based models. While methods such as Naive Bayes and SVM evaluate text features based on basic statistical properties, Transformer-based models benefit from the richness of contextual information thanks to the attention mechanism and large-scale pre-trained language models. In the implementation and experiment phase, each model was trained and tested on the datasets. The results showed that Transformer models generally achieved higher accuracy rates compared to traditional methods, but traditional methods still had an advantage in terms of computational costs and training time. In addition, the classification success of different models varies according to the dataset and problem type; for example, the RoBERTa model achieved the highest accuracy rate on the AG News and SST-2 datasets, while the BERT model stood out on the IMDb dataset. The findings obtained in this thesis are of a guiding nature in model selection for future NLP applications and aim to contribute to both academic and industrial applications. The results also highlight the strengths as well as limitations of the Transformer architecture and provide a basis for developing more efficient and effective methods.
This thesis covers the comparative analysis of traditional methods (Naive Bayes and Support Vector Machines) developed for text classification problems in the field of Natural Language Processing (NLP) and Transformer-based models (BERT, RoBERTa, GPT, T5, DistilBERT) that have shown great success in recent years. The main purpose of the study is to measure the classification performances of these methods on different data sets and to examine the obtained results in detail in terms of accuracy, loss rates and general model success. Within the scope of the thesis, AG News, IMDb and SST-2 datasets were used; text data was subjected to appropriate pre-processing processes and integrated into both traditional and Transformer-based models. While methods such as Naive Bayes and SVM evaluate text features based on basic statistical properties, Transformer-based models benefit from the richness of contextual information thanks to the attention mechanism and large-scale pre-trained language models. In the implementation and experiment phase, each model was trained and tested on the datasets. The results showed that Transformer models generally achieved higher accuracy rates compared to traditional methods, but traditional methods still had an advantage in terms of computational costs and training time. In addition, the classification success of different models varies according to the dataset and problem type; for example, the RoBERTa model achieved the highest accuracy rate on the AG News and SST-2 datasets, while the BERT model stood out on the IMDb dataset. The findings obtained in this thesis are of a guiding nature in model selection for future NLP applications and aim to contribute to both academic and industrial applications. The results also highlight the strengths as well as limitations of the Transformer architecture and provide a basis for developing more efficient and effective methods.
Açıklama
Anahtar Kelimeler
Bilgisayar Mühendisliği Bilimleri-Bilgisayar ve Kontrol, Computer Engineering and Computer Science and Control
