The System is envisioned to use a variety of different voice, visual, automated sensors, machine to machine, and direct tactile inputs and outputs. These are all designed to reduce labor requirements and errors, while improving the test, baselines, ease-of-use, and the overall end user experience. Basically, the System is using technology to replicate (and improve) a person’s vision, hearing, speech, memory, and intelligent correlation.

 

This page covers some of the audio tools and services using embedded microphones in standard PC/Laptops, Smartphones, Tablets, and emerging devices. This page is primarily focused on the Audible aspects of the System, but it does cover some of the other areas where Audio and Visual tools and services can be used in System.

 

Some of the high-level Visual Tools and Services include:

• Localization/Internationalization to present and interact in the users native language, and adapting to some of their cultural and regional differences

• Translation Services

• System and Device Voice Control, Communication, and Monitoring

• Questionnaire/Reminder Logs/Questions/Schedule

• Messages Board for the blind

• Memory (and memory retention) Tests  - linked to schedule - Customizable by the family

• Remote Presence

• Robotics

• Robotic HealthMate

• Active and Passive Examinations

 

Localization/Internationalization/Translation

 

One of the important and critical aspects of the System is to provide the interface to the System and all of the Components in the individual’s native language, and to the extent possible adjust for any cultural and regional differences.  This is important to increase the use of the System to other areas where there could be language barriers, plus allow less trained individuals to perform more tasks.  An example on the Medical Side of the System, it can aid in talking to patients, collecting and storing information, getting instructions, looking up information, consulting with another professional, and/or telemedicine.

 

The System is envisioned to support:

• Many different languages through the use of Objected Oriented Language Support which provides different sets of visual and audio objects depending on a configuration parameter to provide the interface in the caregivers native language

• Support for real-time cross-language translation of verbal to text, text to verbal, and verbal to verbal

• The support of Abstract Interfaces to support certain cultural and regional differences

 

System and Device Voice Control, Communication, and Monitoring

 

Hands free interfaces to devices and the System are important to promote the use of the system to support a wide range of disabilities, as well as provide natural language interfaces.

 

The System is envisioned to support:

• A set of features that can be used by Device Developers to be able to directly communicate and take commands from an individual

• Voice interfaces into the different System Controller functions (services, requests, etc.)

• Monitoring for certain events (help, broken glass, etc.) to improve the safety of the individual

• Questionnaire/Reminder Logs/Questions/Schedule (similar to (if not) the same type of interfaces to today’s Smartphones and Tablets)

 

Note: This can also be provided by a remotely located individual

Messages Board for the Blind

 

Memory (and Memory Retention) Tests 

 

Memory needs to be reinforced, as well as tested for several different ailments that can affect us. Some can be commercial- off-the-shelf, while others can and should be customized for thee individual using sight and sound. These can be automated or manually initiated tests used in everyday life, and/or part of a regularly scheduled examination or “remote check-in”.

 

The System is envisioned to support:

• Two-way voice interface to the tests

• Customizable questions, sound, pictures, and/or video (created through a APP, probably by a guardian) with ability to try to help or escalate for follow-up

• Customizable memory reinforcement

• Standard Memory retention tests

• Memory strengthening puzzles and exercises

 

Note: A Guardian or Call Center could check in with the individual and either have the individual go through one with them

 

Remote Presence/Robotics/Robotic HealthMate

 

Remote presence is a set of technologies that enable a person to have a human-like presence in another, possibly distant location through the use of a remote presence device. The system is envisioned to support this feature and it could be built into a number of robotic devices that could help autonomously, or be the host for Remote Presence as needed.

 

The System is envisioned to support:

• Remote Presence within a dedicated or shared devices

• Verbal, visual, and machine-to-machine interface with robotic and other devices

 

Active and Passive Exams

 

One of the main drivers for the System is to drive down the cost of Healthcare and reducing errors. Enabling native language interfaces into Intelligent Adaptive Examination Decision Trees, translation services, and the use of automatic data collection accomplishes that goal.

The System is envisioned to support:

• Adaptive Examination Decision Trees supporting instructions and data collection different languages, regions, and cultures

• Translation Services to better support talking to patients, collecting and storing information, getting instructions, looking up information, consulting with another professional, and/or telemedicine

• Automated collection, analysis, storage, and comparison of current baseline data (hearing, lungs, heart, etc.) <machine-to-machine as much as possible>

• Remote Presence/Telemedicine to check on and consult on/with patients

 

Notes:

• Audio Tools and Services related to automate Medical Analysis will look for identifiable issues either due to something they can directly detect, or detecting changes by comparing current audio with previous baseline images and/or combine the information with audio input.

• While a lot of potential issues can be identified through Audio Analysis, Audio Analysis is one dimension, and any newly identified items would typically be followed up with questions applicable to the condition through the Intelligent Adaptable Examination Decision Tree (e.g. pain – if so what kind, how much, more at certain times), itch, how long they had it, texture description, color, etc.), and/or with additional testing or taking of samples. For existing known issues, there could be questions as to how it may have changed (similar to the questions if it was a new issue). Detailed baselines are important, and the System creates and stores baselines during examinations.

• A specialized tricorder or wireless stethoscope may need to be developed to be able to hear detailed sounds of the heart, lunch, and joints.  The focus is on having them interface to tablets and smartphones to be able to view, hear, and store the data for analysis.