Joshua Rapp

Joshua Rapp
Joshua's Quick Links

  • Biography

    Josh's research lies at the intersection of optics, electronics, signal processing, and computer vision. His doctoral thesis investigated probabilistic models to improve the performance of single-photon lidar under real-world conditions. Prior to joining MERL, Josh was a postdoctoral researcher at Stanford University. He received a Best Student Paper award at the IEEE International Conference on Image Processing (ICIP) in 2018 and the IEEE SPS Young Author Best Paper award in 2020. His current research interests include computational imaging, statistical signal processing, and active sensing methods.

  • Recent News & Events

    •  EVENT    MERL Contributes to ICASSP 2024
      Date: Sunday, April 14, 2024 - Friday, April 19, 2024
      Location: Seoul, South Korea
      MERL Contacts: Petros T. Boufounos; François Germain; Chiori Hori; Sameer Khurana; Toshiaki Koike-Akino; Jonathan Le Roux; Hassan Mansour; Kieran Parsons; Joshua Rapp; Anthony Vetro; Pu (Perry) Wang; Gordon Wichern
      Research Areas: Artificial Intelligence, Computational Sensing, Machine Learning, Robotics, Signal Processing, Speech & Audio
      Brief
      • MERL has made numerous contributions to both the organization and technical program of ICASSP 2024, which is being held in Seoul, Korea from April 14-19, 2024.

        Sponsorship and Awards

        MERL is proud to be a Bronze Patron of the conference and will participate in the student job fair on Thursday, April 18. Please join this session to learn more about employment opportunities at MERL, including openings for research scientists, post-docs, and interns.

        MERL is pleased to be the sponsor of two IEEE Awards that will be presented at the conference. We congratulate Prof. Stéphane G. Mallat, the recipient of the 2024 IEEE Fourier Award for Signal Processing, and Prof. Keiichi Tokuda, the recipient of the 2024 IEEE James L. Flanagan Speech and Audio Processing Award.

        Jonathan Le Roux, MERL Speech and Audio Senior Team Leader, will also be recognized during the Awards Ceremony for his recent elevation to IEEE Fellow.

        Technical Program

        MERL will present 13 papers in the main conference on a wide range of topics including automated audio captioning, speech separation, audio generative models, speech and sound synthesis, spatial audio reproduction, multimodal indoor monitoring, radar imaging, depth estimation, physics-informed machine learning, and integrated sensing and communications (ISAC). Three workshop papers have also been accepted for presentation on audio-visual speaker diarization, music source separation, and music generative models.

        Perry Wang is the co-organizer of the Workshop on Signal Processing and Machine Learning Advances in Automotive Radars (SPLAR), held on Sunday, April 14. It features keynote talks from leaders in both academia and industry, peer-reviewed workshop papers, and lightning talks from ICASSP regular tracks on signal processing and machine learning for automotive radar and, more generally, radar perception.

        Gordon Wichern will present an invited keynote talk on analyzing and interpreting audio deep learning models at the Workshop on Explainable Machine Learning for Speech and Audio (XAI-SA), held on Monday, April 15. He will also appear in a panel discussion on interpretable audio AI at the workshop.

        Perry Wang also co-organizes a two-part special session on Next-Generation Wi-Fi Sensing (SS-L9 and SS-L13) which will be held on Thursday afternoon, April 18. The special session includes papers on PHY-layer oriented signal processing and data-driven deep learning advances, and supports upcoming 802.11bf WLAN Sensing Standardization activities.

        Petros Boufounos is participating as a mentor in ICASSP’s Micro-Mentoring Experience Program (MiME).

        About ICASSP

        ICASSP is the flagship conference of the IEEE Signal Processing Society, and the world's largest and most comprehensive technical conference focused on the research advances and latest technological development in signal and information processing. The event attracts more than 3000 participants.
    •  
    •  TALK    [MERL Seminar Series 2023] Dr. Kristina Monakhova presents talk titled Robust and Physics-informed machine learning for low light imaging
      Date & Time: Tuesday, November 28, 2023; 12:00 PM
      Speaker: Kristina Monakhova, MIT and Cornell
      MERL Host: Joshua Rapp
      Research Areas: Computational Sensing, Computer Vision, Machine Learning, Signal Processing
      Abstract
      • Imaging in low light settings is extremely challenging due to low photon counts, both in photography and in microscopy. In photography, imaging under low light, high gain settings often results in highly structured, non-Gaussian sensor noise that’s hard to characterize or denoise. In this talk, we address this by developing a GAN-tuned physics-based noise model to more accurately represent camera noise at the lowest light, and highest gain settings. Using this noise model, we train a video denoiser using synthetic data and demonstrate photorealistic videography at starlight (submillilux levels of illumination) for the first time.

        For multiphoton microscopy, which is a form a scanning microscopy, there’s a trade-off between field of view, phototoxicity, acquisition time, and image quality, often resulting in noisy measurements. While deep learning-based methods have shown compelling denoising performance, can we trust these methods enough for critical scientific and medical applications? In the second part of this talk, I’ll introduce a learned, distribution-free uncertainty quantification technique that can both denoise and predict pixel-wise uncertainty to gauge how much we can trust our denoiser’s performance. Furthermore, we propose to leverage this learned, pixel-wise uncertainty to drive an adaptive acquisition technique that rescans only the most uncertain regions of a sample. With our sample and algorithm-informed adaptive acquisition, we demonstrate a 120X improvement in total scanning time and total light dose for multiphoton microscopy, while successfully recovering fine structures within the sample.
    •  

    See All News & Events for Joshua

  • Awards

    •  AWARD    Joshua Rapp wins Best Dissertation Award from the IEEE Signal Processing Society
      Date: December 20, 2021
      Awarded to: Joshua Rapp
      MERL Contact: Joshua Rapp
      Research Areas: Computational Sensing, Signal Processing
      Brief
      • Joshua Rapp has won the 2021 Best PhD Dissertation Award from the IEEE Signal Processing Society.
        The award recognizes a PhD thesis completed on a signal processing subject within the past three years for its relevant work in signal processing while stimulating further research in the field.

        Dr. Rapp completed his PhD at Boston University in 2020 with a thesis entitled "Probabilistic Modeling for Single-Photon Lidar." The dissertation tackles challenges of the acquisition and processing of 3D depth maps reconstructed from time-of-flight data captured one photon at a time.
        The award will be presented at the 2022 IEEE International Conference on Image Processing (ICIP) in France.
    •  
    See All Awards for MERL

  • Internships with Joshua

    • ST0068: Internship - Single-Photon Lidar Algorithms

      The Computational Sensing Team at MERL is seeking an intern to work on estimation algorithms for single-photon lidar. The ideal candidate would be a PhD student with a strong background in statistical modeling, estimation theory, computational imaging, or inverse problems. The intern will collaborate with MERL researchers to design new lidar reconstruction algorithms, conduct simulations, and prepare results for publication. A detailed knowledge of single-photon detection, lidar, and Poisson processes is preferred. Hands-on optics experience is beneficial but not required. Strong programming skills in Python or MATLAB are essential. The duration is anticipated to be at least 3 months with a flexible start date.

    See All Internships at MERL

  • MERL Publications

    •  Sholokhov, A., Nabi, S., Rapp, J., Brunton, S., Kutz, N., Boufounos, P.T., Mansour, H., "Single-pixel imaging of spatio-temporal flows using differentiable latent dynamics", IEEE Transactions on Computational Imaging, October 2024.
      BibTeX TR2024-151 PDF
      • @article{Sholokhov2024oct,
      • author = {{Sholokhov, Aleksei and Nabi, Saleh and Rapp, Joshua and Brunton, Steven and Kutz, Nathan and Boufounos, Petros T. and Mansour, Hassan}},
      • title = {Single-pixel imaging of spatio-temporal flows using differentiable latent dynamics},
      • journal = {IEEE Transactions on Computational Imaging},
      • year = 2024,
      • month = oct,
      • url = {https://www.merl.com/publications/TR2024-151}
      • }
    •  Fernandez-Menduina, S., Rapp, J., Mansour, H., Greiff, M., Parsons, K., "Tracking Beyond the Unambiguous Range with Modulo Single-Photon Lidar", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), DOI: 10.1109/​ICASSP48485.2024.10446835, March 2024, pp. 6-10.
      BibTeX TR2024-021 PDF
      • @inproceedings{Fernandez-Menduina2024mar,
      • author = {Fernandez-Menduina, Samuel and Rapp, Joshua and Mansour, Hassan and Greiff, Marcus and Parsons, Kieran},
      • title = {Tracking Beyond the Unambiguous Range with Modulo Single-Photon Lidar},
      • booktitle = {IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
      • year = 2024,
      • pages = {6--10},
      • month = mar,
      • doi = {10.1109/ICASSP48485.2024.10446835},
      • url = {https://www.merl.com/publications/TR2024-021}
      • }
    •  Sholokhov, A., Rapp, J., Nabi, S., Brunton, S., Kutz, N., Mansour, H., "Single-pixel imaging of dynamic flows using Neural ODE regularization", IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), DOI: 10.1109/​ICASSP48485.2024.10447584, March 2024, pp. 2530-2534.
      BibTeX TR2024-024 PDF
      • @inproceedings{Sholokhov2024mar,
      • author = {Sholokhov, Aleksei and Rapp, Joshua and Nabi, Saleh and Brunton, Steven and Kutz, Nathan and Mansour, Hassan},
      • title = {Single-pixel imaging of dynamic flows using Neural ODE regularization},
      • booktitle = {IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
      • year = 2024,
      • pages = {2530--2534},
      • month = mar,
      • publisher = {IEEE},
      • doi = {10.1109/ICASSP48485.2024.10447584},
      • url = {https://www.merl.com/publications/TR2024-024}
      • }
    •  Ma, Y., Rapp, J., Boufounos, P.T., Mansour, H., "A model of spatial resolution uncertainty for Compton camera imaging", International Conference on Advancements in Nuclear Instrumentation Measurement Methods and their Applications (ANIMMA), DOI: 10.1051/​epjconf/​202328810002, January 2024, pp. 10002.
      BibTeX TR2024-005 PDF
      • @inproceedings{Ma2024jan,
      • author = {Ma, Yanting and Rapp, Joshua and Boufounos, Petros T. and Mansour, Hassan},
      • title = {A model of spatial resolution uncertainty for Compton camera imaging},
      • booktitle = {Advancements in Nuclear Instrumentation Measurement Methods and their Applications (ANIMMA)},
      • year = 2024,
      • pages = 10002,
      • month = jan,
      • publisher = {EPJ Web of Conferences, 288},
      • doi = {10.1051/epjconf/202328810002},
      • url = {https://www.merl.com/publications/TR2024-005}
      • }
    •  Kitichotkul, R., Rapp, J., Goyal, V.K., "The Role of Detection Times in Reflectivity Estimation with Single-Photon Lidar", IEEE Journal of Selected Topics in Quantum Electronics, DOI: 10.1109/​JSTQE.2023.3333834, Vol. 30, No. 1, pp. 8800114:1-14, January 2024.
      BibTeX TR2024-003 PDF
      • @article{Kitichotkul2024jan,
      • author = {Kitichotkul, Ruangrawee and Rapp, Joshua and Goyal, Vivek K},
      • title = {The Role of Detection Times in Reflectivity Estimation with Single-Photon Lidar},
      • journal = {IEEE Journal of Selected Topics in Quantum Electronics},
      • year = 2024,
      • volume = 30,
      • number = 1,
      • pages = {8800114:1--14},
      • month = jan,
      • doi = {10.1109/JSTQE.2023.3333834},
      • url = {https://www.merl.com/publications/TR2024-003}
      • }
    See All MERL Publications for Joshua

  • Other Publications

    •  Joshua Rapp, Yanting Ma, Robin Dawson and Vivek Goyal, "High-Flux Single-Photon Lidar", Optica, Vol. 8, No. 1, pp. 30-39, 2021.
      BibTeX External
      • @Article{Rapp2021,
      • author = {Rapp, Joshua and Ma, Yanting and Dawson, Robin and Goyal, Vivek},
      • title = {High-Flux Single-Photon Lidar},
      • journal = {Optica},
      • year = 2021,
      • volume = 8,
      • number = 1,
      • pages = {30--39},
      • url = {https://doi.org/10.1364/OPTICA.403190}
      • }
    •  Joshua Rapp, Robin M A Dawson and Vivek K Goyal, "Dithered depth imaging", Opt. Express, Vol. 28, No. 23, pp. 35143-35157, November 2020.
      BibTeX External
      • @Article{Rapp2020dither,
      • author = {Rapp, Joshua and Dawson, Robin M A and Goyal, Vivek K},
      • title = {Dithered depth imaging},
      • journal = {Opt. Express},
      • year = 2020,
      • volume = 28,
      • number = 23,
      • pages = {35143--35157},
      • month = nov,
      • url = {http://www.opticsexpress.org/abstract.cfm?URI=oe-28-23-35143}
      • }
    •  Joshua Rapp, Charles Saunders, Julián Tachella, John Murray-Bruce, Yoann Altmann, Jean-Yves Tourneret, Stephen McLaughlin, Robin M. A. Dawson, Franco N. C. Wong and Vivek K. Goyal, "Seeing Around Corners with Edge-Resolved Transient Imaging", Nature Communications, Vol. 11, pp. 5929, November 2020.
      BibTeX External
      • @Article{Rapp2020nlos,
      • author = {Rapp, Joshua and Saunders, Charles and Tachella, Julián and Murray-Bruce, John and Altmann, Yoann and Tourneret, Jean-Yves and McLaughlin, Stephen and Dawson, Robin M. A. and Wong, Franco N. C. and Goyal, Vivek K.},
      • title = {Seeing Around Corners with Edge-Resolved Transient Imaging},
      • journal = {Nature Communications},
      • year = 2020,
      • volume = 11,
      • pages = 5929,
      • month = nov,
      • url = {http://www.nature.com/articles/s41467-020-19727-4}
      • }
    •  Joshua Rapp, Julian Tachella, Yoann Altmann, Stephen McLaughlin and Vivek K Goyal, "Advances in Single-Photon Lidar for Autonomous Vehicles: Working Principles, Challenges, and Recent Advances", IEEE Signal Processing Magazine, Vol. 37, No. 4, pp. 62-71, July 2020.
      BibTeX External
      • @Article{Rapp2020spm,
      • author = {Rapp, Joshua and Tachella, Julian and Altmann, Yoann and McLaughlin, Stephen and Goyal, Vivek K},
      • title = {Advances in Single-Photon Lidar for Autonomous Vehicles: Working Principles, Challenges, and Recent Advances},
      • journal = {IEEE Signal Processing Magazine},
      • year = 2020,
      • volume = 37,
      • number = 4,
      • pages = {62--71},
      • month = jul,
      • url = {https://ieeexplore.ieee.org/document/9127841/}
      • }
    •  Joshua Rapp, Yanting Ma, Robin M. A. Dawson and Vivek K. Goyal, "Dead Time Compensation for High-Flux Ranging", IEEE Transactions on Signal Processing, October 2019.
      BibTeX External
      • @Article{Rapp2018c,
      • author = {Rapp, Joshua and Ma, Yanting and Dawson, Robin M. A. and Goyal, Vivek K.},
      • title = {Dead Time Compensation for High-Flux Ranging},
      • journal = {IEEE Transactions on Signal Processing},
      • year = 2019,
      • month = oct,
      • url = {https://ieeexplore.ieee.org/document/8705308/}
      • }
    •  Joshua Rapp, Robin M. A. Dawson and Vivek K Goyal, "Estimation From Quantized Gaussian Measurements: When and How to Use Dither", IEEE Transactions on Signal Processing, Vol. 67, No. 13, pp. 3424-3438, July 2019.
      BibTeX
      • @Article{Rapp2018d,
      • author = {Rapp, Joshua and Dawson, Robin M. A. and Goyal, Vivek K},
      • title = {Estimation From Quantized Gaussian Measurements: When and How to Use Dither},
      • journal = {IEEE Transactions on Signal Processing},
      • year = 2019,
      • volume = 67,
      • number = 13,
      • pages = {3424--3438},
      • month = jul
      • }
    •  Joshua Rapp, Yanting Ma, Robin M. A. Dawson and Vivek K Goyal, "Dead Time Compensation for High-Flux Depth Imaging", ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2019, pp. 7805-7809.
      BibTeX External
      • @Inproceedings{RappMa2019a,
      • author = {Rapp, Joshua and Ma, Yanting and Dawson, Robin M. A. and Goyal, Vivek K},
      • title = {Dead Time Compensation for High-Flux Depth Imaging},
      • booktitle = {{ICASSP} 2019 - 2019 {IEEE} {International} {Conference} on {Acoustics}, {Speech} and {Signal} {Processing} ({ICASSP})},
      • year = 2019,
      • pages = {7805--7809},
      • month = may,
      • url = {https://ieeexplore.ieee.org/document/8683805/}
      • }
    •  Joshua Rapp, Robin M. A. Dawson and Vivek K Goyal, "Improving Lidar Depth Resolution With Dither", Proceedings of the IEEE International Conference on Image Processing, October 2018, pp. 1553-1557.
      BibTeX External
      • @Inproceedings{Rapp2018b,
      • author = {Rapp, Joshua and Dawson, Robin M. A. and Goyal, Vivek K},
      • title = {Improving Lidar Depth Resolution With Dither},
      • booktitle = {Proceedings of the {IEEE} {International} {Conference} on {Image} {Processing}},
      • year = 2018,
      • pages = {1553--1557},
      • month = oct,
      • url = {https://ieeexplore.ieee.org/document/8451528/}
      • }
    •  Joshua Rapp and Vivek K Goyal, "A Few Photons Among Many: Unmixing Signal and Noise for Photon-Efficient Active Imaging", IEEE Transactions on Computational Imaging, Vol. 3, No. 3, pp. 445-459, September 2017.
      BibTeX External
      • @Article{Rapp2017,
      • author = {Rapp, Joshua and Goyal, Vivek K},
      • title = {A Few Photons Among Many: Unmixing Signal and Noise for Photon-Efficient Active Imaging},
      • journal = {IEEE Transactions on Computational Imaging},
      • year = 2017,
      • volume = 3,
      • number = 3,
      • pages = {445--459},
      • month = sep,
      • url = {https://ieeexplore.ieee.org/document/7932527/}
      • }