Submitted Successfully!
To reward your contribution, here is a gift for you: A free trial for our video production service.
Thank you for your contribution! You can also upload a video entry or images related to this topic.
Version Summary Created by Modification Content Size Created at Operation
1
Nikos Karacapilidis
 -- 2868 2023-07-05 14:07:18 |
2 format
Jason Zhu
 Meta information modification 2868 2023-07-07 05:28:34 |

Video Upload Options

Do you have a full video?
Send video materials Upload full video

Confirm

Are you sure to Delete?
Yes No
Cite
If you have any further questions, please contact Encyclopedia Editorial Office.
Giarelis, N.; Mastrokostas, C.; Karacapilidis, N. Abstractive vs. Extractive Summarization. Encyclopedia. Available online: https://encyclopedia.pub/entry/46460 (accessed on 18 May 2024).
Giarelis N, Mastrokostas C, Karacapilidis N. Abstractive vs. Extractive Summarization. Encyclopedia. Available at: https://encyclopedia.pub/entry/46460. Accessed May 18, 2024.
Giarelis, Nikolaos, Charalampos Mastrokostas, Nikos Karacapilidis. "Abstractive vs. Extractive Summarization" Encyclopedia, https://encyclopedia.pub/entry/46460 (accessed May 18, 2024).
Giarelis, N., Mastrokostas, C., & Karacapilidis, N. (2023, July 05). Abstractive vs. Extractive Summarization. In Encyclopedia. https://encyclopedia.pub/entry/46460
Giarelis, Nikolaos, et al. "Abstractive vs. Extractive Summarization." Encyclopedia. Web. 05 July, 2023.
Abstractive vs. Extractive Summarization
Edit
This entry is adapted from the peer-reviewed paper 10.3390/app13137620

Due to the huge and continuously growing size of the textual corpora existing on the Internet, important information may go unnoticed or become lost. At the same time, the task of summarizing these resources by human experts is tedious and time consuming. This necessitates the automation of the task. Natural language processing (NLP) is a multidisciplinary research field, merging aspects and approaches from computer science, artificial intelligence and linguistics; it deals with the development of processes that semantically and efficiently analyze vast amounts of textual data. Text summarization (TS) is a fundamental NLP subtask, which has been defined as the process of the automatic creation of a concise and fluent summary that captures the main ideas and topics of one or multiple documents.

text summarization deep learning language models abstractive summarization extractive summarization

1. Introduction

Due to the huge and continuously growing size of the textual corpora existing on the Internet, important information may go unnoticed or become lost. At the same time, the task of summarizing these resources by human experts is tedious and time consuming [1]. This necessitates the automation of the task. Natural language processing (NLP) is a multidisciplinary research field, merging aspects and approaches from computer science, artificial intelligence and linguistics; it deals with the development of processes that semantically and efficiently analyze vast amounts of textual data. Text summarization (TS) is a fundamental NLP subtask, which has been defined as the process of the automatic creation of a concise and fluent summary that captures the main ideas and topics of one or multiple documents [2].
Diverse TS applications already exist, including: (i) literature summarization, aiming to handle a long document such as a book, a scientific article, or similar literature resources [3][4]; (ii) news outlet summarization, aiming to summarize information from one or multiple news outlet portals [5][6]; (iii) e-mail summarization [7][8]; (iv) legal document summarization, focusing on the extraction of important aspects from lengthy legal documents [9][10][11]; (v) social media summarization, where social media posts from multiple users are summarized in order to measure the social impact of a certain topic (this application is highly relevant to the field of opinion mining) [12][13]; (vi) argument summarization, as a means for meaningfully aggregating the public opinion in a digital democracy platform [14].
Recently published works outlined a series of insights about diverse TS characteristics. For instance, Ref. [15] assessed various techniques used by extractive summarization approaches, as well as the associated evaluation metrics; Ref. [16] presented an overview of TS datasets, approaches, and evaluation schemas; Ref. [17] elaborated on a comprehensive classification schema of TS approaches, based on their underlying techniques, and performed a comparative assessment of their performance through the use of various metrics; Ref. [18] focused on extractive TS approaches, evaluation metrics, and their limitations; Ref. [19] offered a comprehensive survey of TS and keyword extraction, the latter being a sibling task of TS. In addition, Refs. [2][20] are two comprehensive surveys of TS applications, approaches, datasets, and evaluation techniques, also reporting on associated limitations and challenges. Contrary to earlier works, Ref. [21] reported a comprehensive survey that focused only on abstractive TS approaches, taking into account recent deep learning approaches, while also presenting their comparative evaluation using various versions of the ROUGE metric [22].
The above works, however, have a series of limitations: (i) only a few of them [17][20][21] evaluated the approaches under consideration through a common evaluation framework (e.g., the ROUGE metric); (ii) only a few of them discussed deep learning approaches [2][20][21]; (iii) they did not use alternative evaluation metrics, which yield interesting results, e.g., BLEU [23], as discussed in [22][23][24]; (iv) they did not provide links to the code repositories of their experimental setups and datasets.
Several works reported the importance of developing a comprehensive evaluation framework [2][16][19][21]. Specifically, Ref. [2] stressed the need for the proposal of new solutions regarding the automatic evaluation of TS approaches, while [16] pointed out that the automatic evaluation of TS approaches remains a very promising research area with many open issues. Some of these issues include: (i) the lack of metrics that take into account the mismatch of synonymous terms between human assigned summaries and machine generated ones, (ii) the lack of datasets with quality summaries, and (iii) the lack of datasets for evaluation of multilingual approaches.

2. Extractive Approaches

The goal of extractive approaches is to extract the most important sentences of the document under consideration. These are assembled into a concise summary that captures the most significant aspects of the original text. Various algorithms have been proposed for extractive summarization, each utilizing different techniques for the sentence ranking and extraction step, including: (i) statistical ones, which utilize statistical metrics such as word or sentence frequency; (ii) graph-based ones, which model the document into a graph of sentences, and then utilize graph theory concepts (e.g., centrality, community detection measures, etc.), and (iii) semantic-based ones, which model sentences and their terms into a co-occurrence matrix, which is then analyzed using distributional semantics [25]. In this context, this subsection discusses some of the most prominent approaches in extractive summarization, namely Luhn, LSA, TextRank, LexRank, PositionRank, and TopicRank.
Luhn [26] is one of the earliest approaches in extractive summarization. It utilizes statistical analysis to rank each sentence of a given text, based on the frequency of the most important words and their relative position in that sentence. The highest scoring sentences are extracted to form the final summary. However, this approach has a limitation, as it only focuses on individual words and does not consider the relationship between words or sentences.
Latent semantic analysis (LSA) was one of the earliest techniques used in an attempt to model the semantic relationships between words and capture key concepts in a document [27]. For the task of TS, the work of [28] proposed the LSA technique, which models a document as a term-sentence matrix that represents the frequency of each word in each sentence of the document. Then, it applies singular value decomposition (SVD) to extract the most important semantic features of the document in order to rank and extract the most important sentences. However, some drawbacks of this approach concern the dimensionality and the selection of sentences. To address them, Ref. [29] built a semantic-based approach using LSA that also used more advanced algorithms. Despite such improvements, summarization approaches built on LSA can be computationally expensive, especially for larger texts, due to the use of SVD [30].
The use of graph-based algorithms is another extractive summarization approach that addresses some limitations of earlier approaches since it performs fast and scalable summarizations. One of the earliest and most prominent graph-based ranking approaches is TextRank [31]. The first step of this approach is the representation of the document as a weighted graph of sentences. The sentences of the document are represented as nodes and the relationships between them as edges. A connection between two sentences indicates that there is similarity between them, measured as a function of their overlapping content. After the graph is created, the PageRank centrality algorithm [32] is applied to rank each sentence based on its connections to the other ones. Finally, the top-ranked sentences are selected to form a summary of the input document. The number of extracted sentences can be set as a user-defined parameter for the termination of the algorithm.
LexRank [33] is another graph-based algorithm that relies on PageRank. Its key difference is that each sentence is represented as a vector of the TF-IDF (term frequency—inverse document frequency) scores of the words it contains, while the relationship between these sentence vectors is measured using cosine similarity. A similarity matrix is created with each sentence represented as a row and column, and the elements of the matrix are computed as the cosine similarity score between the sentence vectors. Only similarities above a given threshold are included. To rank the sentences, PageRank is applied. The number of selected sentences can be set similarly to TextRank. Other graph-based approaches that build on TextRank are TopicRank [34] and PositionRank [35]. TopicRank uses a topic-modelling technique, which clusters sentences with similar topics and extracts the most important sentences of each cluster. PositionRank considers both the distribution of term positions in a text and the term frequencies in a biased PageRank, to rank sentences.
Many word embedding models have been developed since the introduction of the pioneering Word2Vec model [36]. Their goal is to capture semantic information for textual terms, thus increasing the accuracy of various NLP tasks. These embeddings are calculated for each term, and their mean vector representation is the document embedding. Recent advancements in deep learning allow the inference of sentence embeddings [37] from pretrained language models, while achieving better accuracy than earlier models.

3. Abstractive Approaches

The need for abstractive approaches resulted from a major drawback of extractive approaches, which is a lack of readability and coherence of the produced text, since extractive approaches utilize simple heuristics to extract and concatenate the most relevant sentences, without accounting for grammatical or syntactical rules [16]. To generate a fluent and coherent summary, more contextual information about the tokens of the input text is required, thus a family of models that generate new phrases in a similar manner to the paraphrasing process of a human reader is needed [2][17]. Many models for abstractive summarization have already been proposed in the literature. As seen in a recent survey [2], these include graph-based [38], rule-based [39], and semantic modelling [40] approaches. These earlier models, however, do not utilize recent advancements in deep learning, which improve many NLP tasks. Newer abstractive summarization approaches build on deep learning models, including: (i) the convolutional neural networks (CNN) and the recurrent neural networks (RNNs); (ii) LSTM and GRU, which improve the original RNNs and are discussed in [41]. Other neural architectures that are not based on CNNs and RNNs include GAN (generative adversarial networks). Certain works use these to build their abstractive approaches, as described in [42][43]. However, these yield lower evaluation scores (i.e., ROUGE) than recent deep learning models, which rely on the model explained in the next paragraph, as validated in [21].
Transformer [44] is a deep learning model that consists of a series of encoder and decoder layers, which utilize the attention mechanism to model the global dependencies of sequential data [45]. Specifically, the self-attention mechanism assigns different weights to different parts of the input, according to their contextual significance. These are encoded in hidden state layers when generating the output sequence. In addition, Transformer models use multi-head attention, which means that attention is applied in parallel to capture different patterns and relationships of the input data. Transformer uses the encoder-decoder model, which encodes information into hidden layers and then decodes it to generate output. These models are semisupervised, due to their unsupervised pretraining on large datasets, followed by supervised finetuning. Approaches built on this model achieve state-of-the-art performance on various text generation tasks, including abstractive summarization. Recent surveys [20][21] discussed and evaluated the differences between earlier abstractive approaches, including those that utilize deep learning models proposed before the introduction of the Transformer architecture.
T5 [46], which stands for text-to-text transfer transformer, is an approach that closely follows the Transformer architecture. It provides a general framework which converts multiple NLP tasks into sequential text-to-text ones. To address each task, it uses a task-specific prefix before the given sequence in the input. The pretraining process comprises both supervised and unsupervised training. The unsupervised objective of the approach includes masking random spans of tokens with unique sentinel tokens. The “corrupted” sentence is passed to the encoder, while the decoder learns to predict the dropped-out tokens on the output layer. A follow up approach, namely mT5 [47], builds on T5 to provide multilingual pretrained baseline models, which can be further finetuned to address diverse downstream tasks in multiple natural languages.
BART [48], which stands for bidirectional auto-regressive transformers, is a multitask deep learning approach, with abstractive summarization being included in them. BART utilizes a “denoising” autoencoder that learns the associations between a document and its “corrupted” form using various textual transformations. These include random token masking or deletion, text infilling, sentence permutation, and document rotation. This autoencoder is implemented as a sequence-to-sequence model with a bidirectional encoder and a left-to-right autoregressive decoder. For its pretraining, it optimizes a reconstruction loss (cross-entropy) function, where the decoder generates tokens found in the original document with higher probability.
PEGASUS [7], which stands for pretraining with extracted gap-sentences for abstractive summarization, is a deep learning approach pretrained solely for the downstream task of abstractive summarization. It introduces a novel pretraining objective for Transformer-based models, called gap sentences generation (GSG). This objective is specifically designed for the task of abstractive text summarization, as it involves the masking of whole sentences, rather than smaller text spans used in previous attempts. By doing so, it creates a “gap” in the input document, where the model is then trained to complete, by considering the rest of the sentences. Another key advantage of this approach is the selection of the masked sentences by utilizing a technique that ranks sentences based on their importance in the document rather than randomly, as suggested in earlier approaches.
Considering the rapidly increasing size and computational complexity of large pretrained models, as noted in [49], researchers were prompted to explore methods to compress them into smaller versions that maintain high accuracy and faster inference in terms of execution time. One such example is the work of [50] that proposes various comprehension techniques, including: (i) direct knowledge distillation (KD), which allows the knowledge transfer between a large model, referred to as the “teacher” model, into a smaller and “distilled” model, referred to as the “student” model; (ii) pseudo-labels, which replace the ground truth target documents of the student model with those of the teacher, and (iii) shrink and finetune (SFT), which shrinks the teacher model to student size by copying a subset of layers and then the finetuning student model again. They also provide various “distilled” pretrained model versions of large pretrained ones, produced by the BART and PEGASUS approaches.

4. Datasets

CNN/Daily Mail [51] is a dataset containing over 300,000 news articles from CNN and the Daily Mail newspaper, written between 2007 and 2015. This dataset is distributed in three major versions. The first one was made for the NLP task of question answering and contains 313 k unique news articles and close to 1 M questions. The second version was restructured for the task of TS; the data in this version are anonymized. The third version provides a nonanonymized version of the data, where individuals’ names can be found in the dataset. Each article is accompanied by a list of bullet point summaries, which abstractively summarize a key aspect of the article. The CNN/Daily Mail dataset has 3 splits: training (92%, 287,113 articles), validation (4.3% 13,368 articles), and test (3.7%, 11,490 articles).
XSum (standing for eXtreme Summarization) is a dataset that provides over 220,000 BBC news articles covering various topics [6]. Each article is accompanied by a one-sentence summary written by a human expert, who for the most part was the original author of the article. XSum has 3 splits: training (90%, 204,045 articles), validation (5%, 11,332 articles), and test (5%, 11,334 articles).
SAMSum [52] is a dataset that contains more than 16,000 online chat conversations written by linguists. These conversations cover diverse topics and formality styles including emoticons, slang words, and even typographical errors. They are also annotated with short third-person summaries, explaining the dialogue between different people.
Reddit TIFU [53] is a dataset consisting of 123,000 Reddit posts from the /r/tifu online discussion forum. These posts are informal stories that include a short summary, which is the title of the post, and a longer one, known as the “TL;DR” (too long; didn’t read) summary.
BillSum [54] is a dataset that deals with US Congressional (USC) and California (CA) state bill summarization. This corpus contains legislation documents from five to twenty thousand characters. In total, it contains 22,200 USC (18,949 train documents and 3269 test documents) and 1200 CA state bills (1237 test documents), accompanied by summaries written by human experts. The data are collected from the US Publishing Office Govinfo and the CA legislature’s website.

5. Evaluation Metrics

For evaluation purposes, researchers employ two well-known families of metrics, namely BLEU and ROUGE. In general, both assess the number of matching n-grams (sequence of terms) between the machine generated summary and the human assigned one. BLEU [23], standing for bilingual evaluation understudy, is an automatic evaluation method that was originally created for the task of machine translation, but can also be applied for the automatic text summarization task, as suggested in [22][23][24]. It is based on the precision metric, which measures the number of words from the machine-generated (candidate) sentence that match the words in the human-written (reference) sentence, divided by the total number of words in the candidate sentence. Specifically, it utilizes a brevity penalty for short sentences and a modified precision. This precision calculates the geometric average of n-gram precisions, while penalizing word repetition. Two variations of BLEU, namely BLEU-1 and BLEU-2, use unigram precision and both unigram and bigram precisions, respectively.
ROUGE [22], which stands for recall-oriented understudy for gisting evaluation, was inspired by the success of the n-gram overlap measure utilized by BLEU. In contrast to BLEU, ROUGE was introduced as a recall-oriented metric. Specifically, the most common setups are: (i) ROUGE-1 and ROUGE-2 for unigrams and bigrams, respectively; (ii) ROUGE-L, which utilizes the longest common subsequence (LCS) between the reference and the machine-generated summary. It is noted that a variation of ROUGE-L, called ROUGE-LSUM, is computed on the summary-level (contrary to ROUGE-L that is computed on the sentence-level).
 

References

  1. Gupta, V.; Lehal, G.S. A Survey of Text Summarization Extractive Techniques. J. Emerg. Technol. Web Intell. 2010, 2, 258–268.
  2. El-Kassas, W.S.; Salama, C.R.; Rafea, A.A.; Mohamed, H.K. Automatic Text Summarization: A Comprehensive Survey. Expert Syst. Appl. 2021, 165, 113679.
  3. Yasunaga, M.; Kasai, J.; Zhang, R.; Fabbri, A.R.; Li, I.; Friedman, D.; Radev, D.R. ScisummNet: A Large Annotated Corpus and Content-Impact Models for Scientific Paper Summarization with Citation Networks. Proc. AAAI Conf. Artif. Intell. 2019, 33, 7386–7393.
  4. An, C.; Zhong, M.; Chen, Y.; Wang, D.; Qiu, X.; Huang, X. Enhancing Scientific Papers Summarization with Citation Graph. Proc. AAAI Conf. Artif. Intell. 2021, 35, 12498–12506.
  5. Hong, K.; Conroy, J.; Favre, B.; Kulesza, A.; Lin, H.; Nenkova, A. A Repository of State of the Art and Competitive Baseline Summaries for Generic News Summarization. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), Reykjavik, Iceland, 26–31 May 2014; European Language Resources Association (ELRA): Luxembourg; pp. 1608–1616.
  6. Narayan, S.; Cohen, S.B.; Lapata, M. Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, 31 October–4 November 2018; Association for Computational Linguistics: Stroudsburg, PA, USA; pp. 1797–1807.
  7. Zhang, J.; Zhao, Y.; Saleh, M.; Liu, P. PEGASUS: Pre-Training with Extracted Gap-Sentences for Abstractive Summarization. In Proceedings of the 37th International Conference on Machine Learning, Virtual Event. 21 November 2020; PMLR. pp. 11328–11339.
  8. Zhang, S.; Celikyilmaz, A.; Gao, J.; Bansal, M. EmailSum: Abstractive Email Thread Summarization. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online. 1–6 August 2021; Association for Computational Linguistics: Stroudsburg, PA, USA; pp. 6895–6909.
  9. Polsley, S.; Jhunjhunwala, P.; Huang, R. CaseSummarizer: A System for Automated Summarization of Legal Texts. In Proceedings of the COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations, Osaka, Japan, 11–16 December 2016; The COLING 2016 Organizing Committee. pp. 258–262.
  10. Kanapala, A.; Pal, S.; Pamula, R. Text Summarization from Legal Documents: A Survey. Artif. Intell. Rev. 2019, 51, 371–402.
  11. Bhattacharya, P.; Hiware, K.; Rajgaria, S.; Pochhi, N.; Ghosh, K.; Ghosh, S. A Comparative Study of Summarization Algorithms Applied to Legal Case Judgments. In Advances in Information Retrieval, Proceedings of the 41st European Conference on IR Research, ECIR 2019, Cologne, Germany, 14–18 April 2019; Azzopardi, L., Stein, B., Fuhr, N., Mayr, P., Hauff, C., Hiemstra, D., Eds.; Springer International Publishing: Cham, Switzerland, 2019; pp. 413–428.
  12. Sun, S.; Luo, C.; Chen, J. A Review of Natural Language Processing Techniques for Opinion Mining Systems. Inf. Fusion 2017, 36, 10–25.
  13. Hu, Y.H.; Chen, Y.L.; Chou, H.L. Opinion Mining from Online Hotel Reviews—A Text Summarization Approach. Inf. Process. Manag. 2017, 53, 436–449.
  14. Adamides, E.; Giarelis, N.; Kanakaris, N.; Karacapilidis, N.; Konstantinopoulos, K.; Siachos, I. Leveraging open innovation practices through a novel ICT platform. In Human Centred Intelligent Systems, Proceedings of KES HCIS 2023 Conference. Smart Innovation, Systems and Technologies, Rome, Italy, 14–16 June 2023; Springer: Rome, Italy, 2023; Volume 359.
  15. Nenkova, A.; McKeown, K. Automatic Summarization. Found. Trends Inf. Retr. 2011, 5, 103–233.
  16. Saggion, H.; Poibeau, T. Automatic Text Summarization: Past, Present and Future. In Multi-Source, Multilingual Information Extraction and Summarization; Poibeau, T., Saggion, H., Piskorski, J., Yangarber, R., Eds.; Theory and Applications of Natural Language Processing; Springer: Berlin/Heidelberg, Germany, 2013; pp. 3–21. ISBN 9783642285691.
  17. Gambhir, M.; Gupta, V. Recent Automatic Text Summarization Techniques: A Survey. Artif. Intell. Rev. 2017, 47, 1–66.
  18. Moratanch, N.; Chitrakala, S. A Survey on Extractive Text Summarization. In Proceedings of the 2017 International Conference on Computer, Communication and Signal Processing (ICCCSP), Chennai, India, 10–11 January 2017; pp. 1–6.
  19. Bharti, S.K.; Babu, K.S. Automatic Keyword Extraction for Text Summarization: A Survey. arXiv 2017, arXiv:1704.03242.
  20. Mridha, M.F.; Lima, A.A.; Nur, K.; Das, S.C.; Hasan, M.; Kabir, M.M. A Survey of Automatic Text Summarization: Progress, Process and Challenges. IEEE Access 2021, 9, 156043–156070.
  21. Alomari, A.; Idris, N.; Sabri, A.Q.M.; Alsmadi, I. Deep Reinforcement and Transfer Learning for Abstractive Text Summarization: A Review. Comput. Speech Lang. 2022, 71, 101276.
  22. Lin, C.Y. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out; Association for Computational Linguistics: Barcelona, Spain, 2004; pp. 74–81.
  23. Papineni, K.; Roukos, S.; Ward, T.; Zhu, W.J. BLEU: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics; Association for Computational Linguistics, Philadelphia, PA, USA, 7–12 July 2002; pp. 311–318.
  24. Graham, Y. Re-Evaluating Automatic Summarization with BLEU and 192 Shades of ROUGE. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, 17–21 September 2015; Association for Computational Linguistics: Stroudsburg, PA, USA; pp. 128–137.
  25. Rieger, B.B. On Distributed Representation in Word Semantics; International Computer Science Institute: Berkeley, CA, USA, 1991.
  26. Luhn, H.P. The Automatic Creation of Literature Abstracts. IBM J. Res. Dev. 1958, 2, 159–165.
  27. Deerwester, S.; Dumais, S.T.; Furnas, G.W.; Landauer, T.K.; Harshman, R. Indexing by Latent Semantic Analysis. J. Am. Soc. Inf. Sci. 1990, 41, 391–407. Available online: https://search.crossref.org/?q=Indexing+by+latent+semantic+analysis+Scott+Deerwester&from_ui=yes (accessed on 30 May 2023).
  28. Gong, Y.; Liu, X. Generic Text Summarization Using Relevance Measure and Latent Semantic Analysis. In Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, New York, NY, USA, 1 September 2001; Association for Computing Machinery: New York, NY, USA; pp. 19–25.
  29. Steinberger, J.; Jezek, K. Using Latent Semantic Analysis in Text Summarization and Summary Evaluation. Proc. ISIM 2004, 4, 8.
  30. Yeh, J.Y.; Ke, H.R.; Yang, W.P.; Meng, I.H. Text Summarization Using a Trainable Summarizer and Latent Semantic Analysis. Inf. Process. Manag. 2005, 41, 75–95.
  31. Mihalcea, R.; Tarau, P. TextRank: Bringing Order into Text. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, Barcelona, Spain, 25–26 July 2004; Association for Computational Linguistics: Stroudsburg, PA, USA; pp. 404–411.
  32. Page, L.; Brin, S.; Motwani, R.; Winograd, T. The Pagerank Citation Ranking: Bring Order to the Web; Technical Report; Stanford University: Stanford, CA, USA, 1998.
  33. Erkan, G.; Radev, D.R. LexRank: Graph-Based Lexical Centrality as Salience in Text Summarization. J. Artif. Int. Res. 2004, 22, 457–479.
  34. Bougouin, A.; Boudin, F.; Daille, B. TopicRank: Graph-Based Topic Ranking for Keyphrase Extraction. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, Nagoya, Japan, 14–19 October 2013; Asian Federation of Natural Language Processing: Singapore; pp. 543–551.
  35. Florescu, C.; Caragea, C. PositionRank: An Unsupervised Approach to Keyphrase Extraction from Scholarly Documents. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vancouver, CMA, Canada, 30 July–4 August 2017; Association for Computational Linguistics: Stroudsburg, PA, USA; pp. 1105–1115.
  36. Mikolov, T.; Chen, K.; Corrado, G.; Dean, J. Efficient Estimation of Word Representations in Vector Space. arXiv 2013, arXiv:1301.3781.
  37. Reimers, N.; Gurevych, I. Sentence-BERT: Sentence Embeddings Using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China, 3–7 November 2019; Association for Computational Linguistics: Stroudsburg, PA, USA; pp. 3982–3992.
  38. Ganesan, K.; Zhai, C.; Han, J. Opinosis: A Graph Based Approach to Abstractive Summarization of Highly Redundant Opinions. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling 2010), Beijing, China, 23–27 August 2010; Coling 2010 Organizing Committee. pp. 340–348.
  39. Genest, P.E.; Lapalme, G. Fully Abstractive Approach to Guided Summarization. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Jeju Island, Korea, 8–14 July 2012; Association for Computational Linguistics: Stroudsburg, PA, USA; pp. 354–358.
  40. Khan, A.; Salim, N.; Farman, H.; Khan, M.; Jan, B.; Ahmad, A.; Ahmed, I.; Paul, A. Abstractive Text Summarization Based on Improved Semantic Graph Approach. Int. J. Parallel. Prog. 2018, 46, 992–1016.
  41. LeCun, Y.; Bengio, Y.; Hinton, G. Deep Learning. Nature 2015, 521, 436–444.
  42. Rekabdar, B.; Mousas, C.; Gupta, B. Generative Adversarial Network with Policy Gradient for Text Summarization. In Proceedings of the 2019 IEEE 13th International Conference on Semantic Computing (ICSC), Newport Beach, CA, USA, 30 January–1 February 2019; pp. 204–207.
  43. Yang, M.; Li, C.; Shen, Y.; Wu, Q.; Zhao, Z.; Chen, X. Hierarchical Human-Like Deep Neural Networks for Abstractive Text Summarization. IEEE Trans. Neural Netw. Learn. Syst. 2021, 32, 2744–2757.
  44. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention Is All You Need. In Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA, 4–9 December 2017; Curran Associates, Inc.: Red Hook, NY, USA, 2017; Volume 30.
  45. Bahdanau, D.; Cho, K.; Bengio, Y. Neural Machine Translation by Jointly Learning to Align and Translate. arXiv 2016, arXiv:1409.0473.
  46. Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; Liu, P.J. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. J. Mach. Learn. Res. 2020, 21, 1–67.
  47. Xue, L.; Constant, N.; Roberts, A.; Kale, M.; Al-Rfou, R.; Siddhant, A.; Barua, A.; Raffel, C. MT5: A Massively Multilingual Pre-Trained Text-to-Text Transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online. 8 June 2021; Association for Computational Linguistics: Stroudsburg, PA, USA; pp. 483–498.
  48. Lewis, M.; Liu, Y.; Goyal, N.; Ghazvininejad, M.; Mohamed, A.; Levy, O.; Stoyanov, V.; Zettlemoyer, L. BART: Denoising Sequence-to-Sequence Pre-Training for Natural Language Generation, Translation, and Comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online. 10 July 2020; Association for Computational Linguistics: Stroudsburg, PA, USA; pp. 7871–7880.
  49. Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. Language Models Are Few-Shot Learners. In Proceedings of the Advances in Neural Information Processing Systems, Online. 6–12 December 2020; Curran Associates, Inc.: Red Hook, NY, USA, 2020; Volume 33, pp. 1877–1901.
  50. Shleifer, S.; Rush, A.M. Pre-Trained Summarization Distillation. arXiv 2020, arXiv:2010.13002.
  51. Hermann, K.M.; Kocisky, T.; Grefenstette, E.; Espeholt, L.; Kay, W.; Suleyman, M.; Blunsom, P. Teaching Machines to Read and Comprehend. In Proceedings of the Advances in Neural Information Processing Systems, Montreal, CMA, Canada, 7–12 December 2015; Curran Associates, Inc.: Red Hook, NY, USA, 2015; Volume 28.
  52. Gliwa, B.; Mochol, I.; Biesek, M.; Wawer, A. SAMSum Corpus: A Human-Annotated Dialogue Dataset for Abstractive Summarization. In Proceedings of the 2nd Workshop on New Frontiers in Summarization, Hong Kong, China, 4 November 2019; Association for Computational Linguistics: Stroudsburg, PA, USA; pp. 70–79.
  53. Kim, B.; Kim, H.; Kim, G. Abstractive Summarization of Reddit Posts with Multi-Level Memory Networks. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA, 2–7 June 2019; Association for Computational Linguistics: Stroudsburg, PA, USA; pp. 2519–2531.
  54. Kornilova, A.; Eidelman, V. BillSum: A Corpus for Automatic Summarization of US Legislation. In Proceedings of the 2nd Workshop on New Frontiers in Summarization, Hong Kong, China, 4 November 2019; Association for Computational Linguistics: Stroudsburg, PA, USA; pp. 48–56.
More
© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.
Upload a video for this entry
Information
Subjects: Computer Science, Artificial Intelligence
Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register :
Nikolaos Giarelis
,
Charalampos Mastrokostas
,
Nikos Karacapilidis
View Times: 519
Revisions: 2 times (View History)
Update Date: 07 Jul 2023
Table of Contents
    1000/1000
    Hot Most Recent
    Feedback