Abstract

Massive volume of textual information, eg news, over the Internet results in the problem of information explosion. Direct reading of massive text is impractical if not incomprehensible. For this reason, there is a growing demand for automatic summarization technology in the digital era. Traditionally, the goal of automatic summarization is to identify relevant excerpts from the target document(s), digest them and represent them in a succinct form for easy reading. While research in summarization has been in good progress in past decades, its effectiveness has been proved extremely low for social media applications, such as microblogging in twitter, WeChat, etc. This is mainly due to word limitation of microblog messages, which in turn leads to lack of proper grammatical structures and discourse information. We investigate how conventional NLP techniques can be best applied to summarizing microblogs. Resembling a document in a traditional natural language text, we first cluster related microblogs to form a microblog document (m-doc). Within a m-doc, microblog paragraphs (m-para) are formed by the leading topics. Practically, a m-para is a microblog re-post tree. Each m-para is comprised of inter-related short microblog messages ('sentences'), which is referred to as m-sent. Also, links between m-sent and m-para are viewed as contextual information. In our research, we propose a new discourse integration technique over m-doc for microblog summarization purpose.