Deep Transformer-based Representation for Text Chunking

Q: How can I download an article?

To download an article from SID, first log in to the site, search for the article title, and click on the 'Download Article' option.

Q: How can I download an ISI article?

To download an ISI article on SID, enter the keyword or article title in the search bar, view the relevant results, click on the desired article, and select the 'Download Article' option.

Q: How can I access the SID database?

To access the SID database, visit SID.ir, create an account, and log in to access scientific resources.

Q: Is downloading articles from SID free?

Some articles on SID are available for free, while others require payment. Details are specified on the article's page.

Kavehzadeh Parsa; Abdollah Pour Mohammad Mahdi; Momtazi Saeedeh

Scientific Information Database (SID) - Trusted Source for Research and Academic Resources

Journal Paper

Paper Information

Journal: Journal of Information Systems and Telecommunication Year:2023 | Volume:11 | Issue:43 Page(s): 176-184

Download Full-Text

View:

Download:

Cites:

Information Journal Paper

Title

Deep Transformer-based Representation for Text Chunking

Author(s)

Kavehzadeh Parsa | Abdollah Pour Mohammad Mahdi | Momtazi Saeedeh | Issue Writer Certificate

Keywords

Text ChunkingQ4

Sequence labelingQ4

Contextualized Word RepresentationQ4

Deep learningQ4

TransformersQ4

Abstract

Text Chunking is one of the basic tasks in natural language processing. Most proposed models in recent years were employed on chunking and other Sequence labeling tasks simultaneously and they were mostly based on Recurrent Neural Networks (RNN) and Conditional Random Field (CRF). In this article, we use state-of-the-art transformer-based models in combination with CRF, Long Short-Term Memory (LSTM)-CRF as well as a simple dense layer to study the impact of different pre-trained models on the overall performance in Text Chunking. To this aim, we evaluate BERT, RoBERTa, Funnel Transformer, XLM, XLM-RoBERTa, BART, and GPT2 as candidates of contextualized models. Our experiments exhibit that all transformer-based models except GPT2 achieved close and high scores on Text Chunking. Due to the unique unidirectional architecture of GPT2, it shows a relatively poor performance on Text Chunking in comparison to other bidirectional transformer-based architectures. Our experiments also revealed that adding a LSTM layer to transformer-based models does not significantly improve the results since LSTM does not add additional features to assist the model to achieve more information from the input compared to the deep contextualized models.

Cites

No record.

References

No record.

Cite

Related Journal Papers

No record.

Related Seminar Papers

No record.

Related Plans

No record.

Recommended Workshops