Okazaki Lab
English

論文輪読（2021）年

このセミナーについて (About this seminar)

担当者が見つけた面白い研究を紹介するセミナーです。以下の内容で構成されます。 In this seminar, presenters share interesting studies that they found. The seminar consists of the following elements.

  1. 論文紹介 (Paper presentation): 面白いと思う論文（基本的にLong Paper）を選んで，その内容を著者の代わりになったつもりで発表する。 A presenter reads a paper (long paper is strongly preffered) that he or she finds interesting, and explains the content as if he or she was an author of the paper.
  2. 著者プレゼンの上映と解説 (Watching talk videos): 担当者は面白いと思う研究の発表動画を選び，その論文の内容を把握する。担当者の補足説明を聞きながら，参加者みんなで発表動画を鑑賞する。 Finding an interesting video presentation, a presenter reads and understands the content of the paper. Attendees watch the presentation video of the author of the paper. The presenter is expected to explain supplementary information about the research.

前者 (1) はプレゼンテーションの練習を兼ねています。後者 (2) は「よい」プレゼンテーションを鑑賞しながら，英語でのプレゼンテーションやディスカッションに慣れることを狙っています。 The former (1) aims at practicing presentations. The latter (2) aims at improving listening and discussion skills in English as we enjoy good presentations.

発表者は一人目は (1) を，二人目は (1) か (2) のどちらかを担当します。 The 1st presentator take (1). The 2nd one can select (1) or (2).

座長(chair)は、セミナーの司会進行をおこないます。 chair person host the seminar. 座長の仕事を参考にして，円滑に議論が進むように心がけてください。 Please contribute to active discussion refering to Chair's Job.

発表時間 (Presentation Time)

  • Presentation: 15 ~ 20 minnutes
  • QA: 5 ~ 10 minutes

日時 (Date and time)

  • 3Q: 12:30~ (Fri)
  • 4Q: 12:30~ (Fri)

参加者 (Attendee)

  • 全員

:exclamation: 発表を登録するときは，著者名，発表年，タイトル，会議名／ジャーナル名，（巻，号，ページ番号など）を必ず記入してください。 :exclamation: Please include author names, publication year, title, conference/journal name, (volume and page numbers) when you add an entry for presentation.

論文へのリンク集 (Links to the papers)

  • ACL Anthlogy: TACL, ACL, NAACL, EMNLP, EACL, COLING, CoNLL, IJCNLP
  • AAAI
  • IJCAI
  • NeurIPS
  • ICML
  • ICLR

著者プレゼンテーションのリンク集 (Links to authors' presentations)

  • ICLR 2021
  • NeurIPS 2020
  • EMNLP 2020
  • ACL 2020
  • ICLR 2020
  • NeurIPS 2019
  • ACL 2019
  • NAACL 2019
  • ICML 2019
  • ICLR 2019
  • NIPS 2018
  • EMNLP 2018
  • ACL 2018
  • NAACL 2018

Tips

  • リモートミーティングでビデオプレゼンテーションを行う場合は，画面およびオーディオを共有してください。
    • Zoomの場合：特別な操作は不要です
    • Microsoft Teamsの場合：システムオーディオの共有(Share the system audio)機能を利用してください。
  • When you stream the video presentation via remote meeting, please share the both screen and system audio.
    • For Zoom: You don't have to do nothing special. Just share your screen.
    • For Microsoft Teams: Please activate Share the system audio feature.

今後の予定 (Planned Seminars)

2021-02-04(Fri) 12:30~

  • presenter
    • Kaneko
    • Yoshikawa
  • chair
    • Ueki

過去の記録 (Past Seminars)

2021-04-14(Wed) 12:30~

  • presenter

    • Koyama

      • Zhaohong Wan, et al. Improving Grammatical Error Correction with Data Augmentation by Editing Latent Representation (COLING 2020)
      • paper, slide

    • Mizuki

      • Tom B. Brown, et al. Language Models are Few-Shot Learners. In: NeurIPS 2020. 2020. (best paper)
      • paper, video, supplementary material
      • Those who are interested in the theoretical background of the effectiveness, I recommend following paper.
        • Saunshi. N, et al. A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks. To Appear: ICLR 2021. 2020. paper, video

2021-04-21(Wed) 12:30~

  • presenter

    • Muraoka

      • Candidate 1: Hao Tan, and Mohit Bansal. Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision. (EMNLP2020)
        • paper -> I will introduce it somewhere else in the near future :)
      • Candidate 2: Irwan Bello. LambdaNetworks: Modeling long-range Interactions without Attention. (ICLR2021, spotlight)
        • paper, slides (4.4 MB)

    • Yang * CapWAP: Captioning with a Purpose
      Adam Fisch, Kenton Lee, Ming-Wei Chang, Jonathan Clark, Regina Barzilay Video, Paper

2021-04-28(Wed) 12:30~

  • presenter _ Taniguchi _ Non-Autoregressive Text Generation with Pre-trained Language Models Yixuan Su, Deng Cai, Yan Wang, David Vandyke, Simon Baker, Piji Li, Nigel Collier EACL2021 * paper, slide

    * Niwa
    * How Can We Know What Language Models Know? (TACL 2020)
        * Zhengbao Jiang, Frank F. Xu, Jun Araki, Graham Neubig
        * [paper](https://www.aclweb.org/anthology/2020.tacl-1.28/), [video](https://virtual.2020.emnlp.org/paper_TACL.1983.html)

2021-05-05(Wed)

  • national holiday

2021-05-12(Wed) 12:30~

  • presenter

    • Maruyama

      • Beta Embeddings for Multi-Hop Logical Reasoning in Knowledge Graphs. In NeurIPS 2020. Slide Paper

    • Liu

      • Martin Schmitt, Sahand Sharifzadeh, Volker Tresp And Hinrich Schütze. An Unsupervised Joint System for Text Generation from Knowledge Graphs and Semantic Parsing. In EMNLP 2020. Video. Paper.

2021-05-19(Wed) 12:30~

  • presenter
    • Wiem
      • UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark [AAAI2021] paper, slides
    • Kuo
      • Joint Constrained Learning for Event-Event Relation Extraction. In EMNLP2020. paper, video

2021-05-26(Wed) 12:30~

  • presenter

    • Hiraoka

      • Error Analysis and the Role of Morphology
      • Marcel Bollmann and Anders Søgaard. EACL2021, Best Paper Award.
      • slide
      • slide with annimation（PRPの参考用）

    • Ueki

      • DEBERTA: DECODING-ENHANCED BERT WITH DISENTANGLED ATTENTION
      • Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen (ICLR2021)
      • slide

    2021-06-02(Wed) 12:30~

  • Presenter:

    • Erick:
      • Nikolay Malkin et all. GPT Perdetry Test: Generating new meanings for new words. In: NAACL 2021.
      • Paper, Slide

  • Sangwhan

    • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
    • Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, Neil Houlsby
    • In ICLR2021.
    • Paper Slides

2021-06-09(Wed) 12:30~

  • Iida

    • Yixing Fan, Jiafeng Guo, Xinyu Ma, Ruqing Zhang, Yanyan Lan, and Xueqi Cheng A Linguistic Study on Relevance Modeling in Information Retrieval WWW2021
    • slides, paper

  • Koyama

    • Dhruv Kumar, Lili Mou, Lukasz Golab, Olga Vechtomova. Iterative Edit-Based Unsupervised Sentence Simplification. ACL 2020.
    • paper, slide

2021-06-16(Wed) 18:05~

  • presenter

    • Mizuki

      • SAUNSHI, Nikunj; MALLADI, Sadhika; ARORA, Sanjeev. A Mathematical Exploration of Why Language Models Help Solve Downstream Tasks. In: ICLR 2021. 2021.
      • paper, slide, video (for the interested readers)

    • Muraoka

      • Hao Tan, and Mohit Bansal. Vokenization: Improving Language Understanding with Contextualized, Visual-Grounded Supervision. (EMNLP2020)
      • video, paper

  • chair

    • Wiem

2021-06-23(Wed) 18:05~

  • presenter
    • Kaneko
      • Cunxiao Du, Zhaopeng Tu, Jing Jiang. Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation (ICML 2021)
      • paper, slides
    • Yoshikawa
      • Evaluation Examples Are Not Equally Informative: How Should That Change NLP Leaderboards?
      • Pedro Rodriguez, Joe Barrow, Alexander Hoyle, John P. Lalor, Robin Jia and Jordan Boyd-Graber (ACL-IJCNLP2021)
      • paper, slides
  • chair
    • Koyama

2021-06-30(Wed) 18:05~

  • presenter

    • Takase

      • Are Pre-trained Convolutions Better than Pre-trained Transformers?
        • Yi Tay, Mostafa Dehghani, Jai Gupta, Dara Bahri, Vamsi Aribandi, Zhen Qin, Donald Metzler. ACL 2021.
        • paper, slide

    • Vijay

      • Hooks in the Headline: Learning to Generate Headlines with Controlled Styles
        • Di Jin, Zhijing Jin, Joey Tianyi Zhou, Lisa Orii, Peter Szolovits. ACL 2020
        • paper, slide

  • chair

    • Ueki

2021-07-07(Wed) 18:05~

  • presenter

    • Liu

      • CoDA: Contrast-enhanced and Diverse-promoting Data Augmentation for Natural Language Understanding. (ICLR 2021)
        • Yanru Qu, Dinghan Shen, Yelong Shen, Sandra Sajeev, Jiawei Han, Weizhu Chen
        • Paper. Slide

    • Yang

  • chair

    • Kuo

2021-07-14(Wed) 18:05~

  • presenter

    • Kuo

      • Automatic Detection of Generated Text isEasiest when Humans are Fooled (ACL 2020)
        • Daphne Ippolito*, Daniel Duckworth*, Chris Callison-Burch, Douglas Eck
        • Paper, Slides(revised)

    • Taniguchi

      • ZeroShotCeres: Zero-Shot Relation Extraction from Semi-Structured Webpages (ACL 2020)
        • Colin Lockard, Prashant Shiralkar, Xin Luna Dong, Hannaneh Hajishirzi
        • Paper, Video

  • chair

    • Maruyama

2021-07-21(Wed) 18:05~

  • presenter

    • Niwa

      • Glancing Transformer for Non-Autoregressive Neural Machine Translation (ACL2021)
        • Lihua Qian, Hao Zhou, Yu Bao, Mingxuan Wang, Lin Qiu, Weinan Zhang, Yong Yu, Lei Li
      • paper, paper (one column ver.), slide

    • Maruyama

      • Ziheng Wang, Jeremy Wohlwend, Tao Lei. Structured Pruning of Large Language Models (emnlp2020)
      • paper, video

  • chair

    • Taniguchi

2021-07-28(Wed) 18:05~

  • presenter
    • Maeda
    • Ueki
      • Meta Back-Translation (ICLR2021)
      • Hieu Pham,, Xinyi Wang, Yiming Yang, Graham Neubig
      • paper, slide
  • chair
    • Toyama

2021-08-04(Wed) 18:05~

  • presenter

    • Ishikawa

      • All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text
        • Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, Noah A. Smith. ACL2021
        • paper, slide

    • Hiraoka

      • Neural Mask Generator: Learning to Generate Adaptive Word Maskings for Language Model Adaptation
      • Minki Kang, Moonsu Han, Sung Ju Hwang. EMNLP 2020.
      • paper, presentation
      • Out of Order: How Important Is The Sequential Order of Words in a Sentence in Natural Language Understanding Tasks?
      • Thang M. Pham, Trung Bui, Long Mai, Anh Nguyen. ACL2021-Findings.
      • slide

  • chair

    • Maeda

2021-09-01(Wed) 18:05~

  • presenter

    • Loem

      • Weight Distillation: Transferring the Knowledge in Neural Network Parameters
      • Ye Lin, Yanyang Li, ZiyangWang, Bei Li, Quan Du, Tong Xiao, Jingbo Zhu. ACL-IJCNLP2021.
      • slide

    • Iida

      • Autoregressive Entity Retrieval Nicola De Cao, Gautier Izacard, Sebastian Riedel, Fabio Petroni
      • video and paper
      • supplement slides

  • chair

    • Ishikawa

2021-09-08(Wed) 18:05~

  • presenter

    • Ma

      • Examining the Inductive Bias of Neural Language Models with Artificial Languages (ACL2021)
      • Jennifer C. White, Ryan Cotterell
      • paper
      • slides

    • Sangwhan

      • HTLM: Hyper-Text Pre-Training and Prompting of Language Models
      • Armen Aghajanyan, Dmytro Okhonko, Mike Lewis, Mandar Joshi, Hu Xu, Gargi Ghosh, Luke Zettlemoyer
      • paper
      • slides

  • chair

    • Loem

2021-09-15(Wed) 18:05~

  • presenter

    • Erick

      • UnNatural Language Inference.
      • Koustuv Sinha, Prasanna Parthasarathi, Joelle Pineau and Adina Williams. In ACL-IJCNLP (2021)
      • paper, slides

    • Takase

      • Efficient Content-Based Sparse Attention with Routing Transformers
        • Aurko Roy, Mohammad Saffar, Ashish Vaswani, David Grangier. TACL 2021.
        • paper, slide

  • chair

    • Ueki

2021-09-22(Wed) 18:05~

  • presenter
    • Maeda
      • Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger
      • Conference : ACL2021
      • Author : Fanchao Qi, Mukai Li, Yangyi Chen, Zhengyan Zhang, Zhiyuan Liu, Yasheng Wang, Maosong Sun
      • [slides] | [paper]
    • Wiem
  • chair
    • Koyama

2021-09-29(Wed) 18:05~

  • presenter

    • Toyama

      • Armen Aghajanyan et al., Intrinsic Dimensionality Explains the effectiveness of Language Model Fine-Tuning (ACL2021)
      • paper, slide

    • Mizuki

  • chair

    • Maruyama

2021-10-08(Fri) 12:30~

  • presenter
    • Muraoka
      • Wei-Ning Hsu, David Harwath, Tyler Miller, Christopher Song, James Glass. "Text-Free Image-to-Speech Synthesis Using Learned Segmental Units." ACL-IJCNLP. 2021.
      • paper, slides (7.7 MB) , demo, poster, video in NeurIPS2020-ws
    • Mizuki
      • MEISTER, Clara; FORSTER, Martina; COTTERELL, Ryan. Determinantal Beam Search. In: ACL2021. 2021.
      • paper, video presentation (requires conference login), presentation by Prof. Miyao at SNLP2021
  • chair
    • Taniguchi

2021-10-15(Fri) 12:30~

  • presenter

    • Taniguchi

      • Daniel Grießhaber, Johannes Maucher
      • Fine-tuning BERT for Low-Resource Natural Language Understanding via Active Learning
      • COLING 2020
      • paper, slide

    • Yang Liu (video presentation)

      • Xinnuo Xu, Guoyin Wang, Young-Bum Kim, Sungjin Lee. AugNLG: Few-shot Natural Language Generation using Self-trained Data Augmentation. ACL-IJCNLP 2021.
      • Paper; Video (requires conference login) ; Slide

  • chair

    • Kuo

2021-10-29(Fri) 12:30~

  • presenter
    • Liu Yang
    • Kuo
      • How Knowledge Graph and Attention Help? A Qualitative Analysis into Bag-level Relation Extraction (ACL-IJCNLP 2021)
        • Zikun Hu, Yixin Cao, Lifu Huang, Tat-Seng Chua
        • paper, slides, video (login required)
  • chair
    • Maeda

2021-11-05(Fri) 12:30~

  • presenter

    • Maruyama Yang

      • Multimodal Transformer for Multimodal Machine Translation Shaowei Yao, Xiaojun Wan ACL 2020 paper multimodal_transformer_multimodal_machine_translation.pdf (646.4 kB)

    • Niwa

      • Probing Pretrained Language Models for Lexical Semantics (EMNLP 2020)
        • Ivan Vulić, Edoardo Maria Ponti, Robert Litschko, Goran Glavaš, Anna Korhonen
        • paper, video, supplementary information

  • chair

    • Loem

2021-11-12(Fri) 12:30~

  • presenter

    • Hiraoka
      • Fast WordPiece Tokenization
      • Xinying Song, Alex Salcianu, Yang Song, Dave Dopson, Denny Zhou
      • EMNLP2021 main. paper, slides.
    • Sangwhan
      • UNKs Everywhere: Adapting Multilingual Language Models to New Scripts
      • Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych and Sebastian Ruder
      • EMNLP2021 main. Paper, Official Slides, Slides

  • chair

    • Ishikawa

2021-11-19(Fri) 12:30~

  • presenter

    • Erick: Predicting emergent linguistic compositions through time: Syntactic frame extension via multimodal chaining. EMNLP 2021

      • Lei Yu, Yang Xu
      • paper
      • Authors presentation
      • Supporting slides

    • Koyama: Point, Disambiguate and Copy: Incorporating Bilingual Dictionaries for Neural Machine Translation

      • Tong Zhang, Long Zhang, Wei Ye, Bo Li, Jinan Sun, Xiaoyu Zhu, Wen Zhao, Shikun Zhang
      • ACL 2021
      • paper
      • slide

  • chair

    • Ueki

2021-11-26(Fri) 12:30~

  • presenter

    • Iida

      • Ultra-High Dimensional Sparse Representations with Binarization for Efficient Text Retrieval (EMNLP2021)

        • Kyoung-Rok Jang, Junmo Kang, , Giwon Hong, , Sung-Hyon Myaeng, Joohee Park, Taewon Yoon, Heecheol Seo

        • paper, refpaper, slides

    • Ueki

      • Prefix-Tuning: Optimizing Continuous Prompts for Generation(ACL-IJCNLP2021)
        • Xiang Lisa Li, Percy Liang
      • paper,author-presentation (youtube)
      • slide

  • chair

    • Maeda

2021-12-03(Fri) 12:30~

  • presenter

    • Loem

      • Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting (EMNLP 2020)
        • Sanyuan Chen, Yutai Hou, Yiming Cui, Wanxiang Che, Ting Liu, Xiangzhan Yu
        • paper | slide

    • Kaneko

      • True Few-Shot Learning with Language Models (NeurIPS 2021)
      • paper, slides

  • chair

    • Koyama

2021-12-10(Fri) 12:30~

  • presenter

    • Yoshikawa

      • (tentative) SituatedQA: Incorporating Extra-Linguistic Contexts into QA (EMNLP2021 main; Outstanding Paper Award)
      • Michael Zhang and Eunsol Choi
      • paper | slides

    • Ma

      • Structured Prediction as Translation between Augmented Natural Languages (ICLR 2021)
      • Giovanni Paolini, Ben Athiwaratkun, Jason Krone, Jie Ma, Alessandro Achille, Rishita Anubhai, Cicero Nogueira dos Santos, Bing Xiang, Stefano Soatto
      • paper | video | supplementary slides

  • chair

    • Taniguchi

2021-12-17(Fri) 12:30~

  • presenter

    • Ishikawa
      • Evidence-based Factual Error Correction
        • James Thorne, Andreas Vlachos. ACL2021
        • paper, slide

  • chair

    • Maruyama

2021-12-24(Fri) 12:30~

  • presenter
    • Maeda [slides] | [paper]
      • Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding (ACL2021)
      • Author : Xin Sun et al.
  • chair
    • Kuo

2021-01-21(Fri) 12:30~

  • presenter _ Vijay _ Aspect-Controllable Opinion Summarization - ACL Anthology * Underline | Aspect-Controllable Opinion Summarization
    • Maruyama
  • chair
    • Loem

2021-01-28(Fri) 12:30~

  • presenter

    • Kuo

      • BARTScore: Evaluating Generated Text as Text Generation (NeurIPS 2021)
      • Weizhe Yuan, Graham Neubig, Pengfei Liu
        • paper, slide

    • Erick: Chao Jia et all. Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision paper video

  • chair

    • Ishikawa