Multi-modal Sensing Co-creation Consortium Industry-academia co-creation platform Joint research promotion program OPERA

R&D Theme

R&D Theme 1: Enhancement of the Sensor Basic Elements

[Summary]
The multimodal sensor's structure is based on the ion image sensor realized by the Toyohashi University of Technology. Conventional manufacturing methods generated several microns of concavity on the sensor surface to ensure sensitivity and operational stability. The concavity is extremely small in our daily life; however, it can reduce the ability to realize high-precision sensors. We have adopted an extended dot-gate structure that prevents the concavity, and we have studied the manufacturing process while ensuring the sensitivity and stability of the sensor device. We are also developing an electronic circuit architecture that enables the sensor to acquire a micron-level spatial resolution and sub-millisecond time resolution.

[Background]
We seek to detect various chemical and physical phenomena through the application of multimodal sensing. In this R&D theme, we will develop Electric Potential Detector Array based on CMOS technology.

[Research & Development]
OPERA_R&D1
[Features]
- Micro-pressure can be detected at the micron level pixel size
- Ion's behavior and pressure can be acquired simultaneously in real-time, and displayed in two-dimensional images
R&D Theme 2: Multi-gas-sensitive Membrane Formation and Sensing Technology

[Summary]
Humans have about 400 different types of olfactory receptors. Olfactory receptors respond to odor characteristics such as molecular structure, polarity, and hydrophilicity. Humans learn and remember the response patterns of the receptors and identify the odor. This mechanism is enabled on a sensor chip with the development of multiple types of odor detection membranes that exhibit changing electrical properties when odor molecules are adsorbed. We also demonstrate the technology to adhere to the membranes to the sensor substrate. Our research goal is to develop more than 30 different odor detection membranes and create coating technology at the millimeter-sized scale. In addition, we will implement AI data processing deep learning technology to replace the human brain's ability to detect odors.

[Background]
To detect multiple gases on a semiconductor sensor, (i) the development of multiple types of odor detection membranes that exhibit changing electrical properties when odor molecules are adsorbed, and (ii) technology to adhere them to the sensor substrate are required.

[Research & Development]
rd02.jpg
[Features]
- Identify various odors using several different odor-sensing membranes.
- Digitalize the odors and generate the odor databases
R&D Theme 3: Piezoelectric Films Development on hydrogen ion-sensitive membrane and its sensing technology

[Summary]
When pressure or physical force is applied to a piezoelectric material, internal polarization occurs, and an electric potential is generated. As developed in R&D Theme 1, a thin piezoelectric film on the sensor can detect pressure and physical force. However, to generate polarization, a high voltage must be applied to the thin piezoelectric film in advance; if the high voltage is applied after the thin piezoelectric film is generated on the electric potential detector array, the sensor is thought to be damaged. We have devised a manufacturing process that has succeeded in detecting small pressures of up to 40 micronewtons.

[Background]
To observe various phenomena of cell activities, we aim to detect the behavior of pressure, pH, and ions within a small area.

[Research & Development]
(Coming soon) re03.jpg
[Features]
- Identification of various odors using different sensitive membranes
- Digitalization of odors to for an odor database
R&D Theme 4: Development of Multimodal Sensing Technology for Environmental Studies

[Summary]
In multi-component gas sensing, many gases of different properties can be detected using various gas receptors; the number of gas receptors that can be produced is not unlimited. Our goal is to add diverse functions to the sensor and realize the high-precision multimodal gas sensing by integrating with big data analysis.

[Background]
In light of terminologies such as "Society 5.0" and "Trillion Sensors" being employed recently, the idea of connecting cyberspace and reality has attracted considerable attention of late. Multimodal sensing technology is essential to make this idea possible, through the measurement of ultra-multidimensional information while maintaining spatio-temporal correlation.

[Research & Development]
Conventional sensor development has focused on high-sensitivity and high-selectivity detection in contaminants and disturbances. On the other hand, in multimodal sensing, the sensor selectivity is not necessarily meaningful. By combining multiple sensors with a different response to the target item, the sensor can capture comprehensive information. Thus, it is a sensing concept suitable for environmental measurements in real space. re04.jpg
[Features]
- Gas measurement in non-ideal environments (e.g., environment with foreign substances, humidity)
- Inspection of various gases through molecular filtration and deep learning
R&D Theme 5: Multimodal Sensing Technology Applied for Agriculture

[Summary]
Plants and crops produce a variety of fragrances, depending on the environment. For example, "green scent" is thought to be released when plants are injured or attacked by pests. The multi-gas sensing technology is expected to detect growth abnormalities by monitoring these scents produced by plants. Also, by monitoring photosynthesis, respiration, and root activity via ion imaging sensor application, we aim to improve agricultural productivity from plant factories to open-air production through cultivation management to maintain optimal growth conditions of the plants.

[Background]
We develop a plant diagnostic technology based on multimodal sensing applied to the food chain of agricultural products, from the production and distribution to food consumption, and ensure the stable supply of fresh vegetables and healthy and nutritious food.

re05-1en.jpg
[Research & Development]
The project applies plant-diagnostic technology with multimodal sensors to the agricultural product's food chain from the production and distribution to food consumption.
(1) Plant diagnostics based on the measurement of odor components
(2) Plant diagnostics for rootsre05-2en.jpg
[Features]
- Accurate estimation of the growth and storage conditions of plants with advanced sensor technologies
- Contribution to healthy and nutritious food through the interactions with sensor-equipped plants
R&D Theme 6: Development of Multimodal Sensing Technology in Medical Drug Discovery

[Summary]
Through the installation of a sensitive membrane that contains substances that allow specific ions to pass through or enzymes to react with specific proteins on the sensor element, we can observe specific behavior of neurotransmitters such as Na, K, and Ca ions and ATP (Adeno Triphosphate) whose concentrations vary depending on the cell activity. This R&D project aims to construct a system to observe the cell activity status and develop a research and development tool that will be useful for medical treatment and drug discovery.

[Background]
We aim to create research and development tools in pharmaceuticals and drug discovery by observing ions behavior around biological materials and cells.

[Research & Development]
re06-en.jpg
[Features]
- Simultaneous observation of neurotransmitters such as K, Na, Ca ions and ATP
- Real-time observation of the living cells behavior caused by unknown viruses and pathogens
R&D Theme 7: Health Monitoring with Multimodal Sensing Technology

[Summary]
This research and development project studies biosensors via the use of molecular imprinting. Molecular imprinting is a technique in which a polymer is polymerized while the target molecule is incorporated. Once the target molecule is removed, and the gap is utilized for molecular recognition. We can design health care devices that can measure the health conditions in a non-invasive manner and not harm the human body with this technology to detect low-concentration biomarkers in human body fluids.

R&D Theme 8: Application of Multimodal Sensing Technology to Human-Machine Harmony

[Summary]
This R&D project explores advanced robotics that can respond to the high health demands of modern society. As the first step, we develop a physical care robot that can monitor a person's pulse and heartbeat to judge the person's mental and physical state and perform subsequent tasks. Multimodal sensing technology aims to establish human-machine harmony that is human-friendly and capable of estimating human emotions.

[Background]
In our world today, people are more concerned about their health and try to improve their quality of life. To this end, co-exist with robots as life partners have become natural.

[Research & Development]
To better understand and control the interaction between manual labor and its effects on the subject, we offer a new physical care robot development and study the technology behind such interactions. Specifically, TEWAZA, an intelligent robot specializing in acupressure and massage movements, is being developed to meet modern society's higher health demand and establish the technology to measure and estimate the effects of robotics on one's physical and mental states.
re08.jpg
[Features]
- Development of robotics that sense and cooperate with humans
- Development of technology for sensing humans and estimate on people's physical and mental state