Home

PerkinsHacks18 wiki!

Challenge:

Designing a solution:

• Design Thinking Approach

• learning from the users, thier expectation, user centric, tailored the solution to incorporate what they needed, e.g. frequency beeps instead of text to speech instructions

• conversational bot can switch between beep and voice commands

• Brainstorming

How it works? - The underlying Idea

• conversational AI
• speech to text to speech system
• computer vision
• mapping a touchscreen into grid co-ordinates
• differentiating static display buttons and digital display [which shows variable output]
• event recognition [actual press on a touchscree should change the display - an event is fired] by detecting changes in the digital panel

How to extend the prototype into a mobile app ?

Overcoming the technical challenges:

• platform independent vision algorithms - so that they can be used from any camera
• algorithms for mapping a screen display
• Hand profiling and hand detection customized to the user, deep learning module for calliberation
• illumination independence, reducing signal to noise ratio, handling reflections
• smooth and more intutive beep generation
• automatic display calliberation and grid map generation using vision and OCR
• allignment, orientation, display recognition on an applicance (like Microsoft's seeing AI)
• proximity and distance analysis for measuring closeness to the digital screen