About Me
I’m a PhD student at the Language Technologies Institute of the School of Computer Science at Carnegie Mellon University. I am advised by Dr. Shinji Watanabe as a member of WAVLab.
I’m generally interested in improving speech & language processing. My current projects involve speech translation and multilingual speech recognition with end-to-end neural networks.
I also completed my master’s degree at CMU SCS where I was advised by Dr. Michael Shamos. Before that, I was a Technology Strategy Consultant at Accenture. I completed my undergraduate degree in Economics and Computer Science at The University of Chicago.
Recent News
- June 2023: Joining the SCALE 2023 Workshop at John’s Hopkins University
- June 2022: Joining the JSALT 2022 Workshop at John’s Hopkins University
- May 2021: Interning at Dr. Dong Yu’s AI lab at Tencent America
Selected Publications
Speech Translation
Cross-Modal Multi-Tasking for Speech-to-Text Translation via Hard Parameter Sharing
Brian Yan, Xuankai Chang, Antonios Anastasopoulos, Yuya Fujita, Shinji Watanabe
2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024
paper
Incremental Blockwise Beam Search for Simultaneous Speech Translation with Controllable Quality-Latency Tradeoff
Peter Polak, Brian Yan, Shinji Watanabe, Alexander Waibel, Ondrej Bojar
Annual Conference of the International Speech Communication Association (INTERSPEECH), 2023
paper
CMU’s IWSLT 2023 Simultaneous Speech Translation System
Brian Yan*, Jiatong Shi*, Soumi Maiti, William Chen, Xinjian Li, Yifan Peng, Siddhant Arora, Shinji Watanabe
Proceedings of the 21th International Conference on Spoken Language Translation (IWSLT), 2023
paper / poster
ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit
Brian Yan*, Jiatong Shi*, Yun Tang, Hirofumi Inaguma, Yifan Peng, Siddharth Dalmia, Peter Polák, Patrick Fernandes, Dan Berrebbi, Tomoki Hayashi, Xiaohui Zhang, Zhaoheng Ni, Moto Hira, Soumi Maiti, Juan Pino, Shinji Watanabe
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), 2023
paper / poster
Align, Write, Re-order: Explainable E2E Speech Translation via Operation Sequence Generation
Motoi Omachi*, Brian Yan*, Siddharth Dalmia, Yuya Fujita, Shinji Watanabe
2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023
paper
CTC Alignments Improve Autoregressive Translation
Brian Yan, Siddharth Dalmia, Yosuke Higuchi, Graham Neubig, Florian Metze, Alan W Black, Shinji Watanabe
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (EACL), 2023
paper / talk / poster / TLDR
CMU’s IWSLT 2022 Dialect Speech Translation System
Brian Yan, Patrick Fernandes, Siddharth Dalmia, Jiatong Shi, Yifan Peng, Dan Berrebbi, Xinyi Wang, Graham Neubig, Shinji Watanabe
Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT), 2022
paper / talk
ESPnet-ST IWSLT 2021 Offline Speech Translation System
Hirofumi Inaguma*, Brian Yan*, Siddharth Dalmia, Pengcheng Guo, Jiatong Shi, Kevin Duh, Shinji Watanabe
Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT), 2021
paper / TLDR
Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Tasks
Siddharth Dalmia, Brian Yan, Vikas Raunak, Florian Metze, Shinji Watanabe
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), 2021
paper / talk
Multilingual Speech Recognition
Improving Massively Multilingual ASR With Auxiliary CTC Objectives
William Chen, Brian Yan, Jiatong Shi, Yifan Peng, Soumi Maiti, Shinji Watanabe
2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023
paper
Towards Zero-Shot Code-Switched Speech Recognition
Brian Yan, Matthew Wiesner, Ondrej Klejch, Preethi Jyothi, Shinji Watanabe
2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023
paper / poster / TLDR
Joint Modeling of Code-Switched and Monolingual ASR via Conditional Factorization
Brian Yan, Chunlei Zhang, Meng Yu, Shi-Xiong Zhang, Siddharth Dalmia, Dan Berrebbi, Chao Weng, Shinji Watanabe, Dong Yu
2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022
paper / talk / poster / TLDR
Differentiable Allophone Graphs for Language-Universal Speech Recognition
Brian Yan, Siddharth Dalmia, David R. Mortensen, Florian Metze, Shinji Watanabe
Annual Conference of the International Speech Communication Association (INTERSPEECH), 2021
paper / talk / TLDR
My Google Scholar is more comprehensive.
Activities
Talks
Controllable and Explainable End-to-End Speech Translation
SIG SLT Seminar, 2022
Code-Switched Modeling
JSALT Workshop, John’s Hopkins University, 2022
Building End-to-End Speech Translation Systems
JSALT Workshop, John’s Hopkins University, 2022
Teaching
CS 11-751: Speech Recognition and Understanding
Teaching Assistant
Carnegie Mellon University, Fall 2023
CS 11-700: Language Technologies Institute Colloquium
Teaching Assistant
Carnegie Mellon University, 2021-22 Academic Year
CS 11-737: Multilingual NLP
Teaching Assistant
Carnegie Mellon University, Spring 2021 DSTA Course
Academic Service
Reviewer
ICASSP, Interspeech, ASRU, EMNLP, NAACL, APSIPA, SLT
Contact
Email: byan[at]cs.cmu.edu