Dr. Thomas T Lu
Senior Researcher
About
Education
Received a Ph.D. degree in Electrical Engineering from the Pennsylvania State University.
Topic Area(s)
- Autonomous Systems | Anomaly Detection And Recovery
- Artificial Intelligence, Machine Learning, and Data Science | Supervised And Unsupervised Learning
- Artificial Intelligence, Machine Learning, and Data Science | Deep Learning And Data Analytics
- Robotic Systems | Sensing, Perception And State Estimation
- Entry, Descent, and Landing Systems | Modeling And Simulation Technologies
Search Keyword(s)
Experience
Professional Experience
2004 – Present: Senior Researcher, Bio-Inspired Technologies & Systems Group, JPL
- Task Manager of “Collaborative Moonwalkers”, a 2023 summer internship project building a 3D virtual test environment for collaborative robotic exploration in lunar south pole. This project employees Nvidia Omniverse virtual environment, Virtual Reality (VR), Augmented Reality (AR), Digital Twin of Rovers, autonomous path planning, AI based Rover team collaboration technologies. A paper has been submitted to IEEE Aerospace Conference. A demonstration video is on YouTube https://www.youtube.com/watch?v=E0Rz0ZbwhJY&t=156s.
- Technical Leader on “Surface Water and Ocean Topography (SWOT) Satellite Data Automatic Data Analytic Tools Development” (2023-2024). This project will use Deep Neural Network (DNN) to extract temporal-spatial features and image processing to automatically clean up and organize terabytes of SWOT data.
- Principal Investigator and Task Manager of “JPL Project Knowledge Graph and Intelligent Search Tool (GIST)” (2023-2024). This project uses LLMs, graph knowledge bases, Graph Neural Networks (GNNs) to construct the knowledge graph of JPL project knowledge and help design/test engineers to find past lessons learned during design/test processes.
- Principal Investigator and Task Manager of “Requirement Alignment (REQAL)” (2021-2023). This project uses LLM, GNNs to construct a JPL Expertise Knowledge Graph for matching JPL talents to R&D solicitations. A REQAL software has been developed under Beta testing.
- Principal Investigator and Task Manager of “Investigating Spacecraft Guidance Navigation and Control (GNC) and Vision Control Element (VCE) Implementation on a Next-Gen Multi-Core Processor” (2023-2025). This project investigates next-generation multi-core processors for high-speed computation of AI/ML algorithms on-board spacecraft.
- Principal Investigator and Task Manager of “Trusted & Explainable AI for Saving Lives (TruePAL)” technology development project funded by Dept. of Transportation (DOT) (2020-2022).
- Successfully demonstrated TruePAL, an AI assistant that helps first responder drivers to avoid crash in and around active traffic in a simulated environment.
- Task Manager of “Anomaly Detection for Mission Control Room” (2019) funded by NASA. This project uses natural language processing (NLP) to transcribe audio data and uses recurrent neural network to analyze time-series data for anomaly detection.
- AI/ML Technical Team Lead of “Assistant for Understanding Data through Reasoning, Extraction and Synthesis (AUDREY)” technology development project that developed an AI assistant using machine learning, computer vision, sensor fusion, AI reasoning to assist first responders in emergency situations for Department of Homeland Security (DHS) (2016-2018).
- Demonstrated the computer vision part of an AI assistant for a 911 Call Center collecting relevant video images, monitoring the progression of fire and track the locations of first responders in Grand County, WA.
- Demonstrated the computer vision part of an AI assistant for a Hazmat team detecting people falling in water in Harris County, TX.
- Demonstrated the AI and computer vision part of an AI assistant guiding paramedics and verifying each step during an entire rescue procedure in Hastings County, Ontario, Canada.
- Task manager of “PRISM multi-band IR image segmentation and measurement” task (2015-2019). Developed deep learning models to accurately detect, identify, segment and measure the IR features of objects in multi-spectral IR video images.
- Technical monitor for 3 NASA SBIR Phase I and one Phase II projects (2019-2023). Guided the technology development of parallel processors for high performance space computing.
- Technical Lead for the development of a deep learning model for predicting oil/gas well production for an oil/gas company (2018-2019). Developed simulation models of gas wells using deep learning and accurately predicted the oil/gas production.
- Sub-task manager of “Holographic Storage Unit (HSU)” task (2012-2018) for the development of the high-speed electronic interface systems. Led the development of a FPGA board for high speed I/O using PCIe interface.
- Designed a high-speed continuous 1000 frames/s HD resolution DLP interface for HSU write-in operations.
- Designed a high-speed continuous 1000 frames/s 4 Mpixel CMOS sensor interface for HSU read-out operations.
- Designed and implemented high-speed data processing and error-correction coding on the GPU boards for the HSU system.
- Designed automatic abnormal pattern detection system using a neural network for Deep Space Network (DSN) radar antenna signal analysis and health monitor (2012-2016). Developed a web-based dashboard monitoring the system health of DSN antennas.
- Designed multi-stage automatic target recognition (ATR) algorithms, funded by Navy (2008-2014).
- Designed ATR systems for hi-res periscope video image analysis and target detection;
- Designed ATR systems for underwater sonar signal processing and target identification;
- Designed signal processing algorithms for rotating field mass spectrometer (RFMS) sensors for under-sea chemical compounds detection.
- Collaborated with universities on microwave radar signal processing for non-destructive testing and vital sign detection (2007-2010);
- Developed Windows and Linux based image analysis software (2004-2008);
- Designed and refined neural network-based target tracking algorithms (2004-2010);
- Designed and refined a 3D imaging system (2004-2006);
- Coordinated the efforts by internal research team and external contractors;
- Involved in hyperspectral imaging, Fourier Transform Spectroscopy instrumentation experiments and other R&D projects.
- Mentored over 100 NASA Fellowship/Internship students;
Research Community Service
Program Committee Member and Session Chair for SPIE International Conference on “Pattern Recognition and Tracking” (2018 – present).
Achievements
Publications
- E. Chow, T. Lu, K. Payumo, G. Bardi de Fourou, E. Sadler, N. Elieh Janvisloo, J. Carrillo, B. Li, B. Hubler, O. Littlejohn, V. Hernandez-Cruz, S. Torrellas, “Collaborative Moonwalkers,” IEEE Aerospace Conference, 2024.
- S. Bian, D. Grandi, T. Liu, P. K. Jayaraman, K. Willis, E. Sadler, T. Lu, R. Otis, N. Ho, B. Li, “HG-CAD: Hierarchical Graph Learning for Material Prediction and Recommendation in CAD,” Journal of Computing & Information Science in Engineering, JCISE-22-1394, 2023.
- T. Lu, “An explainable AI assistant for first responder driver safety,” (Keynote Address), International Conference on Computer and Information Technology (ICCIT), 2022.
- T. Lu, K. Yun, A. Huyen, E. Chow, “TruePAL: an AI assistant for first responder safety ” (Invited Paper), SPIE Proceedings, 12101-19, 2022.
- D. Lundstorm, A. Huyen, A. Mevada, K. Yun, T. Lu, “Explainability Tools Enabling Deep Learning in Future In-Situ Real-Time Planetary Explorations,” IEEE Aerospace Conference, 2022.
- K. Yun, K. Adams, J. Reager, Z. Liu, C. Chavez, M. Turmon, T. Lu, “Remote estimation of geologic composition using interferometric synthetic-aperture radar in California's Central Valley,” Tackling Climate Change with Machine Learning: workshop at NeurIPS, 2022.
- S. Bian, D. Grandi, K Hassani, E. Sadler, B. Borijin, T. Lu, R. Otis, N. Ho, B. Li, “Material Prediction for Design Automation Using Graph Representation Learning,” Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC-CIE DAC-88049, 2022.
- A. Huyen, V. Do, M. Gabriel, J. Joubert, T. Lu, K. Yun, E. Chow, “A virtual assistant for first responders using natural language understanding and optical character recognition,” SPIE Defense + Commercial Sensing Paper 12101-17, 2022.
- K. Yun, T. Lu, A. Huyen, E. Chow, “Neurosymbolic hybrid approach to driver collision warning,” SPIE Proc. 12101-24, 2022.
- L. Lai, A. Blakely, M. Invernizzi, J. Lin, T. Kidambi, K. Melstrom, K. Yu, T. Lu, “Separation of color channels from conventional colonoscopy images improves deep neural network detection of polyps,” Journal of Biomedical Optics, 26(01), 2021
- G. Perera, T. Lu, E. Chow, “TruePAL – an AI Assistant for First Responder Safety,” International Association of Chiefs of Police Conference, 2021 New Orleans, Louisiana, Sept. 2021.
- K. Yun, K. H. Kim, A. Pradhan, J. Reager, Z. Liu, M. Turmon, A. Huyen, T. Lu, V. Chandrasekaran, A. Stuart, “Filling the gap: Estimation of soil composition using InSAR, groundwater depth, and precipitation data in California’s Central Valley,” ESS Open Archive, 2021.
- A. Huyen, K. Yun, S. DeBaun, S. Wiggins, J. Bustos, T. Lu, E. Chow, “Dynamic fire and smoke detection and classification for flashover prediction,” SPIE Proceedings, Pattern Recognition and Tracking Conference, 2021.
- K. Yun, T. Lu, A. Huyen, “Transforming unstructured voice and text data into insight for paramedic emergency service using recurrent and convolutional neural networks,” SPIE Proceedings, Pattern Recognition and Tracking Conference, https://arxiv.org/abs/2006.04946, 2020 .
- G. Perera, E. Chow, T. Lu, “Tech Talk: AI Assistant for First Responder Driver Safety,” https://www.policechiefmagazine.org/tech-talk-ai-assistant-for-first-responder-driver-safety/ , 2020.
- T. Lu, A. Huyen, L. Nguyen, J. Osborne, S. eldin, K. Yun, “Optimized training of deep neural network for image analysis using synthetic objects and augmented reality,” SPIE Proceedings, Pattern Recognition and Tracking XXX, DOI:10.1117/12.2522198, 2019.
- K. Yun, K. Yu, J. Osborne, T. Lu, “Improved visible to IR transformation using synthetic data augmentation with cycle-consistent adversarial network,” SPIE Proc. 2019.
- K. Yun, L. Nguyen, T. Nguyen, T. Lu, E. Chow, “Small target detection for search and rescue operations using distributed deep learning and synthetic data generation,” SPIE Conference, 2019.
- K. Yun, T. Lu, “Transforming unstructured data into insight for anomaly detection in exploration ground systems,” GSAW (Best Presentation Award), 2019.
20. T. Lu and T-H. Chao, “Advances in Pattern Recognition Research,” Editors of book by Nova Science Publishers, ISBN: 978-1-53614-429-1, 2018.
21. T. Lu, K. Payumo, L. Seguin, E. Chow, G. Torres, “Augmented reality data generation for training deep neural network,” SPIE Conference on Pattern Recognition and Tracking XXIX, https://doi.org/10.1117/12.2305202, 2018.
22. T. Lu, A. Huyen, K. Payumo, G. Torres, “Deep neural network for precision multi-band infrared image segmentation,” SPIE Conference on Pattern Recognition and Tracking XXIX, 2018.
23. J. Osborne, M. Lee, K. Yun, T. Lu, E. Chow, “Automatic speech recognition for launch control center communication using recurrent neural networks with data augmentation and custom language model,” Conference on Disruptive Technologies in Information Sciences, May 2018.
24. K. Yun, J. Osborne, M. Lee, T. Lu, E. Chow, “Automatic speech recognition for launch control center communication using recurrent neural networks with data augmentation and custom language model,” SPIE Conference on Pattern Recognition and Tracking XXIX, 2018.
25. K. Yun, T. Lu, E. Chow, “Occluded object reconstruction for first responders with augmented reality glass using conditional generative adversarial networks,” SPIE Conference on Pattern Recognition and Tracking XXIX, 2018.
26. K. Yun, J. Bustos, T. Lu, “Predicting Rapid Fire Growth (Flashover) Using Generative Adversarial Networks, ” Electronic Imaging Conference, https://arxiv.org/abs/1801.09804, 2018.
27. T. Lu, A. Luong, S. Heim, G. Torres, “Intelligent multi-spectral IR image segmentation,” SPIE Defense and Security Conferences, 2017.
28. T. Lu, S. Heim, A. Luong, M. Patel, K. Chen, T-H Chao, E. Chao, G. Torres, “Intelligent multi-spectral IR image segmentation,” (Invited Paper), SPIE Conference on Pattern Recognition and Tracking XXVII, https://doi.org/10.1117/12.2262730, 2017.
29. T. Lu, T-H Chao, K. Chen, A. Leung, M. Dewees, X. Yan, E. Chow, And G. Torres, “Cross‐correlation and image alignment for multi‐band IR sensors”, (Invited Paper), SPIE Proc. Vol. 9845 No. 4, 2016.
30. T-H. Chao, T. Lu, S. Davis, M. Anderson, “Chip scale broadly tunable laser spectrometer,” SPIE Defense and Security Conference, 2016.
31. T. Lu, C. Costello, M. Ginley-Hidinger, T-H. Chao, “Adaptive threshold and error-correction coding for robust data retrieval in optical media,” Proc. SPIE Vol. 9477-21, 2015.
32. T-H. Chao, T. Lu, and G. Reyes, “Real-time Holographic Content Addressable Storage,” SPIE Vol. 9477, 2015.
33. B. Walker, T. Lu, C. Costello, G. Reyes, and T-H Chao, “Addressing channel noise and bit rate in a multi-channel free space optical communication system,” SPIE Proc. Optical Pattern Recognition XXV, Vol. 9094, 2014.
34. K. Vincent, D. Nguyen, B. Walker, T. Lu, and T-H Chao, “GPU processing for parallel image processing and real-time object recognition,” SPIE Proc. Optical Pattern Recognition XXV, Vol. 9094, 2014.
35. J. K. Fitzsimons and T. Lu, “Markov random fields for static foreground surveillance systems,” SPIE Proc. Vol. 9217-59, 2014.
36. T-H. Chao, T. Lu, B. Walker, G. Reyes, “High-speed optical processing using digital micromirror device,” SPIE Proc. Optical Pattern Recognition XXV, Vol. 9094, 2014.
37. H. A. Valdez, T. Lu, T-H Chao, “Small object detection via fast discrete Curvelet transform,” SPIE Proc. Vol. 8857, 2013.
38. S. Pandya, T. Lu, and T-H Chao, “Optimizing feature selection strategy for adaptive object identification in a noisy environment,” SPIE Proceedings Vol. 8662, 2013.
39. B. Walker, T. Lu, Sean Stuart, George Reyes, and Tien-Hsin Chao, “Optical image processing and pattern recognition algorithms for optimal optical data retrieval,” SPIE Proceedings Vol. 8748, 21, 2013.
40. T-H Chao and T. Lu, ”High-speed Optical Correlator with Custom Electronics Interface Design,” SPIE Proc. Vol. 8748, 2013.
41. T-H. Chao, T. Lu, S. Davis, G. Farca, S. D. Rommel, “Compact Liquid Crystal Waveguide Fourier Transform Spectrometer for Real-Time Gas Sensing in NIR Spectral Band,” SPIE Optical Pattern Recognition XXIII, Vol. 8298, 2012.
42. B. Walker, T. Lu, T-H. Chao, “Intelligent Image Analysis for Image-Guided Laser Hair Removal and Skin Therapy,” SPIE BiOS, Vol. 8207-A, 2012.
43. M. Scholten, N. Dhingra, T. Lu, T-H. Chao, “Optimization of Support Vector Machine for Object Recognition,” SPIE Optical Pattern Recognition XXIII, Vol. 8298, 2012.
44. J. Chiang, Y. Zhang, T. Lu, T-H. Chao, “Composite Wavelet Filters for Enhanced Automatic Target Recognition,” SPIE Optical Pattern Recognition XXIII, Vol. 8298, 2012.
45. T. Lu, T. Pham and J. Liao, “Identification of Abnormal System Noise Temperature Patterns in Deep Space Network Antennas Using Neural Network Trained Fuzzy Logic,” The 3rd International Conference on Advances in Satellite and Space Communication (SPACOMM), URI: http://hdl.handle.net/2014/42066 2011.
46. T-H. Chao, T. Lu, “Autonomous learning approach for automatic target recognition processor,” (Invited Paper), SPIE Optical Pattern Recognition XXII, Vol. 8055, 2011.
47. M. R. Boesen, D. Keymeulen, J. Madsen, T. Lu, T-H. Chao, “Integration of the Reconfigurable Self-Healing eDNA Architecture in an Embedded System and Evaluation of it using a Fourier Transform Spectrometer Instrument Application,” IEEE Aerospace Conf., 2011.
48. T-H. Chao, T. Lu, M. R. Boesen, D. Keymeulen, “Monolithic liquid crystal waveguide Fourier transform spectrometer for gas species sensing,” SPIE Optical Pattern Recognition XXII, Vol. 8055, 2011.
49. T-H. Chao, T. Lu, “Feasibility breadboard demonstration of an imaging Fourier transform spectrometer using solid state time delay,” SPIE Optical Pattern Recognition XXII, Vol. 8055, 2011.
50. T-H. Chao, T. Lu, “Accelerated sensor data processing using a multichannel GOC/NN processor,” Proceedings of SPIE Vol. 7696C, 2010.
51. Tsung Han (Hank) Lin, T. Lu, Henry Braun, Western Edens, Yuhan Zhang, T-H. Chao, Christopher Assad, and Terrance Huntsberger, “Optimization of a multi-stage ATR system for small target identification,” Proceedings of SPIE Vol. 7696C, 2010.
52. M. T. Wolf, C. Assad, Y. Kuwata, A. Howard, H. Aghazarian, D. Zhu, T. Lu, A. Trebi-Ollennu, T. Huntsberger, “360-Degree Visual Detection and Target Tracking on an Autonomous Surface Vehicle,” Journal of Field Robotics, 2010.
53. W. N. Greene, Y. Zhang, T. Lu , T-H. Chao, “Feature extraction and selection strategies for automated target recognition,” SPIE Symposium on Defense, Security & Sensing Conference, Independent Component Analyses, Wavelets, Neural Networks, Biosystems, and Nanoengineering, Proceedings of SPIE Vol. 7703, 2010.
54. N. R. Prasad;, S. Almanza-Garcia and T. Lu, “Anomaly Detection,” CMC, Vol. 14, No. 1, 2009.
55. Ye, D., Edens, W., Lu, T., Chao, T., “Neural Network target identification system for false alarm reduction,” Proc. SPIE Vol. 7340, 2009
56. Johnson, O., Edens, W., Lu, T., Chao, T. "Optimization of OT-MACH filter generation for target recognition," Optical Pattern Recognition XX. Proceedings of the SPIE, Volume 7340, 2009.
57. Stephen Williams and T. Lu, “Visual target tracking in the presence of unknown observer motion,” Proc. SPIE, Vol. 7340 -11, Optical Pattern Recognition XX. 2009.
58. T. Lu, T-H. Chao, “A neural network identification system for space-borne GCMS pattern recognition,” Proc. SPIE 6574, SPIE Symposium on Defense and Security, Optical Pattern Recognition XVIII, 2007.
59. T-H. Chao, T. Lu, “System issues of developing grayscale optical correlator for ATR applications,” invited paper, Proc. SPIE 6574, SPIE Symposium on Defense and Security, Optical Pattern Recognition XVIII, 2007.
60. T. Lu, C. Snapp, T-H. Chao, A. Thakoor, T. Bechtel, S. Ivashov, and I. Vasiliev, “Evaluation of holographic subsurface radar for NDE of space shuttle thermal protection tiles,” Proc. SPIE 6555, SPIE Symposium on Defense and Security, Sensors and Systems for Space Applications, 2007.
61. N. R. Prasad, T. Lu, and J. C. King, “ Machine intelligence-based decision-making (MIND) for Automatic Anomaly Detection,” Proc. SPIE 6574, SPIE Symposium on Defense and Security, Optical Pattern Recognition XVIII, https://doi.org/10.1117/12.723635, 2007.
62. T. Lu, T.-H. Chao, “A single-camera system captures high-resolution 3D images in one shot,” http://newsroom.spie.org/x5109.xml?highlight=x533, 2006.
63. T. Lu, T.-H. Chao, “A high-resolution and high-speed 3D imaging system and its application in ATR,” Proc. SPIE 6245, SPIE Symposium on Defense and Security, Optical Pattern Recognition XVII, 2006.
64. T. Lu, C. L. Hughlett, H. Zhou, T-H. Chao, J. C. Hanan, “Neural network post-processing of grayscale optical correlator,” Proc. SPIE 5908, Optical Information Processing III, 2005.
65. J. W. Fronczek, N. R. Prasad, T. Lu, A. Thakoor, “Locating Leaks in Pressurized Environmental Containments: Shockwave Sensing Using Dynamic Sensor Nets”, the 6th International Conference on Intelligent Technologies InTech05, Phuket, Thailand, December 2005.
66. J. C. Hanan, T.-H. Chao, C. Assad, C. L. Hughlett, H. Zhou, T. Lu, “Closed-Loop Automatic Target Recognition and Monitoring System,” Proc. SPIE 5816 , p. 244-251, Optical Pattern Recognition XVI, 2005.
67. T. Lu, J, Zhang, “Three dimensional imaging system,” United States Patent No. 6,252,623, 2001.
68. T. Lu, D. Mintzer, “Hybrid neural networks for nonlinear pattern recognition”, Optical Pattern Recognition, ed. by F. T. S. Yu & S. Jutamulia, Cambridge University Press, 1998.
69. Z. Z. Ho, T. Lu, “Hybrid neural network and multiple fiber probe for in-depth 3-D mapping”, United States Patent No. 5,660,181, 1997.
70. T. Lu, Lerner, J., “Spectroscopy and hybrid neural network analysis,” Proceedings of the IEEE, Volume 84, Issue 6, pp895 – 905, 1996.
71. T. Lu, D. T. Mintzer, A. A. Kostrzewski, F. S. Lin, “Compact holographic optical neural network system for real-time pattern recognition,” Optical Engineering, Volume 35, number 8, 1996.
72. D. Mintzer, S. Zhao, and T. Lu, “Custom neural networks aid spectroscopic analysis,” Environ. Test Mag., vol. 5, no. 1, pp. 32-35, Jan. 1996.
73. J. Lerner, T. Lu, “Binary optical spectrum analyzer,” United States Patent No. 5,461,475, 1995.
74. T. Lu, F. S. Lin, A. Kostrzewski, J. Lerner, “Large-scale holographic neuron system for multispectral sensor fusion and high-speed signal processing,” Proc. SPIE Vol. 2093, p. 390-399, Substance Identification Analytics, 1994.
75. Y. Sheng, T. Lu, D. Roberge, and H. John Caulfield, "Optical N4 Implementation of a Two-Dimensional Wavelet Transform," Optical Engineering 31, 1859 – 1864, 1992.
76. T. Lu, F. T. S. Yu, and D. A. Gregory, “Self-organizing optical neural network for unsupervised learning,” Opt. Eng., vol. 29, 1859-1864, 1992.
77. F. T. S. Yu and T. Lu, “Adaptive Optical System for Neural Computing,” IEEE Region 10 Conference on Computer and Communication Systems, Hong Kong, 1990.
78. T. Lu, X. Xu, S. Wu, and F. T. S Yu, “Neural network model using interpattern association,” Appl. Opt., vol. 29, no. 2, p.284, 1990.