In our course “Blended Interaction”, Master’s students Michael Zöllner and Stephan Huber have been working on a very different approach to use the Microsoft Kinect. Since we liked their project so much and their helmet-mounted Kinect is such an eye-catcher (check out the video! J), we asked them to write about it for our blog. Here is what they wrote:
NAVI (Navigational Aids for the Visually Impaired) is a student project aiming at improving indoor navigation for visually impaired by leveraging the Microsoft Kinect camera, a vibrotactile waistbelt and markers from the AR-Toolkit.While the “white cane” is a good tool to improve navigation for visually impaired, it has certain drawbacks such as a small radius or that it just detects objects that are on the ground (during typical use).We wanted to augment the visually impaired person’s impression of a room or building by providing vibro-tactile feedback that reproduces the room’s layout.
For this, depth information from the Kinect is mapped by our software onto three pairs of Arduino LilyPad vibration motors located at the left, center and right of the waist. These pairs of vibration motors are hot glued into a fabric waist belt and connected to an Arduino 2009 board. To increase the impact of the vibration motor they were put into the cap of a plastic bottle. The Arduino in the waist belt is connected via usb to a laptop that was mounted onto a special backpack-construction, which has holes for cables and fan.
To support point-to-point navigation usually a seeing-eye dog is used. This dog however must be trained for certain routes, costs a lot of money and gets tired soon. In certain research projects GPS is used to provide this point-to-point navigation, however GPS is not applicable for indoor scenarios.
We wanted to utilize the rgb camera of the Kinect, so we put several markers of the AR-Toolkit on the walls and doors of our building thereby modeling a certain route from one room to another. The markers are tracked continuously all along the way and our software provides synthesized auditory navigation instructions for the person. These navigation instructions vary based on the distance of the person to the marker (which we get from Kinect’s depth camera). So for example, if you walk towards a door the output will be “Door ahead in 3”, “2”, “1”, “pull the door” where each part of the information depends on the distance to the marker on the door.
The software is written with C#/.NET. We used the MangedOpenNI (https://github.com/kobush/ManagedOpenNI) wrapper for the Kinect and the managed wrapper of the ARToolkitPlus (http://code.google.com/p/comp134artd) for marker tracking. Voice synthesis is done using Microsoft’s Speech API (http://msdn.microsoft.com/en-us/speech/default). All input streams are glued together using Reactive Extensions for .NET (http://msdn.microsoft.com/en-us/devlabs/ee794896).







12 Comments
Nice device!
I think there’s a mistake in the section about the motors though:
“three pairs of Arduino LilyPad vibration motors”
These are not “Arduino LilyPad”, just LilyPad devices; people think you’ve gone crazy and used 4-7 Arduinos to control the motors….
Btw, you’re on hackaday.com, congrats!
How is the kinect responding to a 10xAA=15 Volts power supply?
The kinect has a nominal 12 V voltage supply.
It should have been down regulated, what happens when the batteries are <12V?
@Jonathan
You’re right. The text might be a bit misleading, however the picture and video points it out very well: We have three pairs of LilyPad vibration motors connected to one single Arduino Board.
@Paolo
We use ten 1.2V accumulators from Varta which gives a total of 12V (in theory). The real voltage of the accumulators is de facto slightly higher, especially when they are fully charged, so sometimes the battery pack gives approximately 13-14V.
We didn’t monitor the voltage of the battery pack all the time, so we have no answer as to what happens when it drops below 12V. We just changed batteries, when the Kinect didn’t work anymore…
Hallo Leute, wir haben schon bei Youtube was dazu geschrieben. Hier aber vielleicht nochmal kurz. Die Idee und die Umsetzung sind gerade für den gedachten Einsatzzweck großartig! Wir haben auf unserer Seite http://www.live4d.de eine große Sammlung an Kinect Hacks und würden uns deshalb freuen, wenn wir von Euch in der Zukunft weitere Infos darüber erhalten würden. Aktuell haben wir es von Engadget.com erfahren.
Bengala inteligente
Para deficientes visuais.
Veja fotos do equipamento no blog
Israel-tagliari.blogspot.com
Como montar é muito simples e de baixo custo +- R$ 100,00 ou u$70,00
Será a tecnologia para facilitar a vida destas pessoas com deficiência visual, vai consistir de uma bengala inteligente ou também poderá ser um sistema parecido com um celular , que será fixo na cintura ou na cinta da pessoa.
Terá um olho com uma câmera fotográfica para poder filma e identificar o que se encontra a sua volta, em um raio de 2 m2, para dar tempo ao deficiente de receber a mensagem, que a filmadora filmara e transmitira através de um som sonoro ou viva voz com menor ou maior intensidade indicando que esta mais próximo ou mais afastado do obstáculo a sua volta .
Por ex: Obstáculos diversos uma construção, um barraco, escada, veiculo, poste etc. também poderá uma voz indicar dizendo o tipo de objeto que esta a sua frente ou sua volta.num raio de 2 metros +-.
Indicando para que lado seguir por voz ou por sons mais ou menos rápido. ou seqüencial.
Será equipada com uma placa codificada com sensor de distancia ou mesmo utilizada nos veículos importados para ser mais fácil.
Um sensor de estacionamento de veículos que é utilizado para facilitar o estacionamento do veiculo para não bater.
Este sensor detecta dispara um alarme sonoro e determina a distancia que esta o obstáculo.
Que pode ser equipada com um micro processador. Ou um chip , ou placa ou mesmo um programa soft.
A montagem desta placa pode ser tanto na bengala ou em um equipamento tipo de um celular ou outra forma a critério da pesquisa dos inventores projetista.
E também poderá ser usado como um rastreador de pessoas com deficiências de memória e que se perde ou se esquece do endereço através de um GPS. Com mapa e viva voz.
Que indica o local que esta e pode ser programado para onde que se locomover. E também poderá ser equipado junto ao celular, este equipamento para deficientes.
Indicando onde moram com todas as informações que estarão gravadas no chip do equipamento da pessoa, que esta usando o identificador e com as informações dos familiares.
Seqüência do equipamento o que deve ser projetado para fazer:
1* terá que fazer a filmagem
2*Informar ao deficiente o que esta a sua frente, e a distancia.
3*Como deve fazer para desviar, para a direita ou esquerda e a distância, em centímetros ou metros.
4* Informação deve ser por viva voz ou sonora.
5* Mapa de localização através de GPS por satélite que possa ser programado para onde quer ir, e poder montar o caminho programando o trajeto.
Ou ser informado pelo próprio programa.
6* Pode também ser um equipamento somente com um sensor de estacionamento de veículos.
7* que somente vai dar um sinal sonoro quando estiver próximo de um obstáculo, que pode ser montado com um sensor com o visor que vai detectar o perigo e indicar a proximidade do objeto.
8* poderá ser montado dentro de uma bolsa pequena que poderá ser fixada na cintura do deficiente, que é ligada a uma ou mais baterias ou pilhas.
E que tenha a volt programada equivalente a 9 ampares, equivalente a uma bateria comum de controle de TV. Para ter menos peso e quantidade de baterias.
É somente necessário direcionar o visor do sensor para ser o olho do deficiente.
9* Custo do sensor no Paraguai R$ 50,00 +- no Brasil você vai pagar o lucro de venda do comerciante que pode variar de preço.
Ou compre pela internet.
Se quiser pode ser encaixado em uma bengala este sensor, e a bateria pode ser fixada na cinta, para diminuir o volume.
10* Como montar é só necessário compra o sensor que já vem completo e adaptar um adaptador para a bateria ou pilhas, por que para os veículos é ligado a bateria que é de 12 v.
Mas com uma bateria de 9 v também funciona o melhor e mudar para 9 v ou colocar duas de 9 v vai durar mais tempo de preferência recarregável que pode ser reutilizada varias vezes.
O adaptador de baterias pode ser comprado em lojas de produtos eletrônicos ou retirado de algum brinquedo que seja com pilhas ou baterias. É somente montar em uma bolsa ou um recipiente que fique preso todo o equipamento do sensor, adaptador e a bateria, pronto é só testar o equipamento se movimentado para frente e para os lados ou para traz.com o visor sempre virado para o local que vai andar.
Atenção fabricantes a dica esta dada é somente fabricar e colocar no mercado utilizando este nicho de mercado e ficar milionário mas sem muita ambição no valor de venda, ganhe na quantidade produzida faça o que os Chineses fazem. Pense no seu semelhante deficiente que não tem a visão que você tem.
E também para ser utilizado em centenas de outras atividades diversas.
Professor Pardal curitiba, Paraná Brasil
e-mail israel.tagliari@yahoo.com.br
Hello nice to meet you , i am Taiwan student i study in Southern Taiwan University(STUT)
, i watch your video NAVI – Navigational Aids for the Visually Impaired – A student project
in the course Blended Interaction] on youtube that is so interesting for me , because i have
been learn Augmented Reality too , and my question is how you use ARToolkit access in to Kinect?
sorry let you, wast your time to anwser my qusetion , Because in my contry Taiwan augmented reality
is not very popular now ,so i didn’t have any book cangive me example or consult it so hope you can
help me thank you very much .
[p.s] i am very to have the honor to looking the video from your study”
I’m doing a presentation on RX at MIX this year and would love to include a picture of your project in the slide deck as creative uses of the framework. Please contact me to let me know if you are willing for me to use an image and link to the project.
could u give a brief description about external power supply to kinect…
Thnx in advance
hallo,
ich habe gerade von eurem projekt auf http://www.spiegel.de/netzwelt/gadgets/0,1518,769765-5,00.html gelesen und es wird geschrieben, das system ist noch nicht ganz ausgereift weil immer ein laptop mitgenommen werden muß – klar, ist groß und umständlich
warscheinlich habt ihr selber schon daran gedacht – aber wie wäre es, den laptop mit einem smartphone bzw. mit einem netbook zu ersetzen? oder ist das ganz abwegig?
How can i get your code?
http://navi.codeplex.com/
Is there anyway that i can get your code? or contact the personwho made thisby mail?
99 Trackbacks
[...] Human-Computer Interaction | Mensch-Computer Interaktion | Usability Engineering | Konstanz. [...]
[...] the last few months, but some students at the [University of Konatanz] have taken Kinect hacking to a whole new level of usefulness. Rather than use it to control lightning or to kick around some boxes using Garry’s Mod, they [...]
[...] the last few months, but some students at the [University of Konstanz] have taken Kinect hacking to a whole new level of usefulness. Rather than use it to control lightning or to kick around some boxes using Garry’s Mod, they [...]
[...] You can read more over at the Human-Computer Interactions blog entry. It’s really amazing what can be done with off-the-shelf components these days. I don’t think I’d believe it if I was told five years ago what we’d have available. [...]
[...] [via HCI Blog @ Universität Konstanz] [...]
[...] now | £129 | Kinect/HCI Blog (via [...]
[...] For more details on the project and to keep up todate with things, check out the official website here. var [...]
[...] Mar 2011 14:07:00 EDT. Please see our terms for use of feeds.Permalink SlashGear | University of Konstanz | Email this | Comments Get [...]
[...] | University of Konstanz | Email this | Comments [...]
[...] [Via: Crunchgear.com - Hci.uni-konstanz.de] [...]
[...] | University of Konstanz | Email [...]
[...] and it's all demonstrated for you below. Dig that hat, man. Dig that hat. SlashGear University of Konstanz Leave A Comment Tweet arduino, diy, germany, hack, kinect, Michael Zöllner, [...]
[...] [University of Konstanz via Engadget] [...]
[...] NAVI – Navigational Aids for the Visually Impaired – A student project in the course Blended Int… (University of Konstanz) [...]
[...] | University of Konstanz | Email [...]
[...] Fuente: Human Computer Integration University of Konstanz Blog [...]
[...] Para más información: NAVI – Navigational Aids for the Visually Impaired – A student project in the course Blended Int… [...]
[...] haciendo cambiar de parecer a MS en el camino. Pero sin lugar a dudas, uno de los mejores es el NAVI (Ayuda de navegacion para débiles visuales) desarrollado en la Universidad de [...]
[...] | University of Konstanz | Email [...]
[...] Group from the university of Konstanz has created Navigational Aids for the Visually Impaired (or NAVI), an application that uses Kinect’s perception of depth to guide visually impaired people in [...]
[...] | University of Konstanz | Email [...]
[...] Group from the university of Konstanz has created Navigational Aids for the Visually Impaired (or NAVI), an application that uses Kinect’s perception of depth to guide visually impaired people in [...]
[...] | University of Konstanz | Email [...]
[...] Via : http://hci.uni-konstanz.de/ Articles Relatifs :MiRG : Facebook et le vol d'identité !! Twxlate : Traduire les tweets en temps réel !! MIRG : outils statistiques de trafic de site web Cameyo : créer votre propre application portable !! PHP Freelancer The Foreign Exchange and Money Markets Guide EB-5 Investment Opportunity, Gold Coast Florida Regional Center, Opens its Doors Let’s Play Hack [3] Bottomless pits are for epic fails! Skill at walking endlessly! MiRG : Ouvrir sa porte d’entrée sans clefs !! [...]
[...] for helping the blind, two students in Germany have rigged up a Kinect to provide audible directions to a sight-impaired user. It’s a very inelegant [...]
[...] NAVI – Navigational Aids for the Visually Impaired – A student project in the course Blended Int… (University of [...]
[...] NAVI – Navigational Aids for the Visually Impaired – A student project in the course Blended Int… (University of [...]
[...] Group from the university of Konstanz has created Navigational Aids for the Visually Impaired (or NAVI), an application that uses Kinect’s perception of depth to guide visually impaired people in [...]
[...] NAVI – Navigational Aids for the Visually Impaired – A student project in the course Blended Int… (University of [...]
[...] NAVI – Navigational Aids for the Visually Impaired – A student project in the course Blended Int… (University of [...]
[...] | University of Konstanz | Email [...]
[...] | University of Konstanz | Email this | Comments Share and [...]
[...] | University of Konstanz | Email [...]
[...] | University of Konstanz | Email [...]
[...] | University of Konstanz | Email [...]
[...] for helping the blind, two students in Germany have rigged up a Kinect to provide audible directions to a sight-impaired user. It’s a very inelegant [...]
[...] | University of Konstanz | Email [...]
[...] for helping the blind, two students in Germany have rigged up a Kinect to provide audible directions to a sight-impaired user. It’s a very inelegant [...]
[...] Kinect hack Konstanz University [...]
[...] Link: NAVI – Navigational Aids for the Visually Impaired – A student project in the course Blended Int… (University of Konstanz) [...]
[...] for helping the blind, two students in Germany have rigged up a Kinect to provide audible directions to a sight-impaired user. It’s a very inelegant [...]
[...] for helping the blind, two students in Germany have rigged up a Kinect to provide audible directions to a sight-impaired user. It’s a very inelegant [...]
[...] | University of Konstanz | Email [...]
[...] | University of Konstanz | Email [...]
[...] http://hci.uni-konstanz.de/blog/2011/03/15/navi/?lang=en [...]
[...] | University of Konstanz | Email [...]
[...] | University of Konstanz | Email [...]
[...] Blog entry from the University of Konstanz: NAVI – Navigational Aids for the Visually Impaired – A student project in the course Ble… [...]
[...] Blog entry from the University of Konstanz: NAVI – Navigational Aids for the Visually Impaired – A student project in the course Ble… [...]
[...] Más información (en inglés y alemán) en el Blog de la Universidad. [...]
[...] and Stephan Huber. Hopefully it will be improved and hit the market in a not-so-far future. More info. If you enjoyed this post, make sure you subscribe to the RSS feed! You can also follow me on [...]
[...] | University of Konstanz | Email [...]
[...] More information: http://hci.uni-kon … avi/?lang=en [...]
[...] More information: http://hci.uni-kon … avi/?lang=en [...]
[...] página oficial del proyecto [...]
[...] Click here to view a video demonstrating the technology and a more detailed explanation of how the device works. Comments (0) Trackbacks Print this post addthis_pub = 'y2kesq'; addthis_options = 'email, favorites, linkedin, twitter, digg, delicious, google, facebook, stumbleupon, reddit, more'; Filed in New Technologies, Patient Education, Video Tagged Kinect, NAVI You might also enjoy…Smartphone’s Blinput Allows Visually Impaired to Attain Greater Independence [...]
[...] by Michael Zöllner and Stephan Huber from the University of Konstanz, NAVI (or Navigational Aids for the Visually Impaired) allows the blind to easily navigate an environment [...]
[...] to Universität Konstanz’s Human-Computer Interaction Blog and [...]
[...] Switched: Created by Michael Zöllner and Stephan Huber from the University of Konstanz, NAVI (or Navigational Aids for the Visually Impaired) allows the blind to easily navigate an environment [...]
[...] by Michael Zöllner and Stephan Huber from the University of Konstanz, NAVI (or Navigational Aids for the Visually Impaired) allows the blind to easily navigate an environment [...]
[...] by Michael Zöllner and Stephan Huber from the University of Konstanz, NAVI (or Navigational Aids for the Visually Impaired) allows the blind to easily navigate an environment [...]
[...] Blog entry from the University of Konstanz: NAVI – Navigational Aids for the Visually Impaired – A student project in the course Blended Int… [...]
[...] en Engadget.com Vía SlashGear Original uni-konstanz.de) SHARETHIS.addEntry({ title: "Kinect se desvela como un elemento interesantísimo como ayuda a [...]
[...] компонентом экспериментальной системы NAVI (Navigational Aids for the Visually Impaired) является беспроводной [...]
[...] en un ordenador público que vaya usando la gente cuando te encuentre por la calle.— Dani Burón [Konstanz Uni] [...]
[...] Muy buena idea que esperemos hagan más portátil y manejable para que pueda usarse con comodidad y sin perder la dignidad, o al menos que no te convierta en un ordenador público que vaya usando la gente cuando te encuentre por la calle.— Dani Burón [Konstanz Uni] [...]
[...] Muy buena idea que esperemos hagan más portátil y manejable para que pueda usarse con comodidad y sin perder la dignidad, o al menos que no te convierta en un ordenador público que vaya usando la gente cuando te encuentre por la calle.— Dani Burón [Konstanz Uni] [...]
[...] It’s sure more towards the multipurpose modify of the bit and digit we wait to wager more of in the future. The recording beneath gives a beatific account of what crapper be achieved. For more info on the send and to ready up todate with things, analyse discover the authorised website here. [...]
[...] all part of the NAVI project (Navigational Aids for the Visually [...]
[...] Muy buena idea que esperemos hagan más portátil y manejable para que pueda usarse con comodidad y sin perder la dignidad, o al menos que no te convierta en un ordenador público que vaya usando la gente cuando te encuentre por la calle.— Dani Burón [Konstanz Uni] [...]
[...] all part of the NAVI project (Navigational Aids for the Visually [...]
[...] artificial vision HCI Blog This uses a head-mounted Kinect camera to spot potential obstacles for a visually impaired person, [...]
[...] Human-Computer Interaction – Konstanz University | [...]
[...] the visually impaired get around without bumping into things. Now, through their project called NAVI (Navigation Aids for the Visually Impaired), students at Germany’s Universität Konstanz [...]
[...] project put together by two Masters students at the University of Konstanz in Germany, the NAVI or Navigation Aids for the Visually Impaired system uses the Kinect to detect objects in the world, alerting the wearer to their [...]
[...] mais informações: Autor PHP Freelancer PHP Freelancer Tweet Categorias: Curiosidades, Hardware, Novas Tecnologias [...]
[...] más información puedes visitar: NAVI. El sitio está en ingles, y es por parte de la [...]
[...] reseachers with the NAVI project at the University of Konstanz have taken the Microsoft Kinect, which provides a rich 3 dimensional visual view of the world to a [...]
[...] 0. http://hci.uni-konstanz.de/blog/2011/03/15/navi/?lang=en 1. http://www.gizmag.com/kinect-as-a-set-of-eyes/18179/ [...]
[...] SEEING EYE HELMET: Engineers from the University of Konstanz in Germany mounted a Kinect camera on a helmet, added a Bluetooth headset, Arduino LilyPad vibration motors on a special belt and a backpack computer. The gear could help a visually impaired user navigate to a specific destination indoors, using feedback to the user through the belt and headset. The Kinect can detect distance to markers placed in the environment. The helmet plus Kinect may not be a hit look, but with refinement this could be a very useful approach. [...]
[...] [hci.uni-konstanz] More related postsJust Another New GPS Device, the Globalsat’s GV-380Nokia To Use Windows Phone While Keeping SymbianDownload Windows Phone 7.0.7390.0 “NoDo” Update (Copy and Paste)Apple Gets Top Rated Brands in the World While Google Remains the Best of AllPreview Release of WebM components for IE9 Available to DownloadInternet Explorer 9 Reaches 2.35 Million Downloads in the first 24 hoursHTC Ignite Windows Phone 7 Leaks SpecsDownload Microsoft Internet Explorer 9 NowDownload Applications for Window Phone 7New Symbian UI Fixes Statusbar, Icons and Navigation [...]
[...] Read | Permalink | Email this | Comments [...]
[...] polygons we throw around, we have a great work from students at the German University of Konstanz. Their system is called NAVI: Navigational Aids for the Visually Impaired. NAVI works something like this. The infrared camera [...]
[...] Read | Permalink | Email this | Comments [...]
[...] Read | Permalink | Email this | Comments [...]
[...] Navigation for People with Visual Impairment [...]
Kinect navigiert Blinde wie ein NAVI…
Menschliche Computer Interaktion könnte der nächste große Schritt sein, dank der Microsoft Bewegungssteuerung Kinect und diversen speziellen Hacks dafür. Fleißige Studenten an der Konstanz Universität haben ein System gebaut, dass Blinden den Weg durch…
[...] polygons we throw around, we have a great work from students at the German University of Konstanz. Their system is called NAVI: Navigational Aids for the Visually [...]
[...] surroundings and give audio navigation commands to guide him or her through the area [source: Zöllner and Huber]. It’s very similar to the way a GPS system in your car directs you to turn left or right at [...]
[...] Erlanger Studentenprojekt beschäftigt sich damit, wie man Kinect zur Indoor-Navigation für Blinde einsetzen kann. Ich muss hier einschränken: kaum [...]
[...] http://www.hci.uni-konstanz.de [...]
[...] http://www.hci.uni-konstanz.de [...]
[...] fascinating projects. Notable examples include a navigational device for the visually impaired and a system for providing surgeons with sensory [...]
[...] fans to tinker with the hardware and come up with fascinating projects. Notable examples include a navigational device for the visually impaired and a system for providing surgeons with sensory feedback during remote [...]
[...] fans to tinker with the hardware and come up with fascinating projects. Notable examples include a navigational device for the visually impaired and a system for providing surgeons with sensory feedback during remote [...]
[...] learn more about the project from the University of Konstanz click on the following link. Human-Computer Interaction. University of Konstanz Computer & Information Science Share [...]
[...] Update: Thanks to a heads up from a commenter, the idea is actually realized with a software project at the University of Konstanz. [...]
[...] dieser Film diese Idee zwar schön aufbereitet und verpackt, aber nicht umsetzt, haben Studenten der Uni Konstanz ein solches System schon 2011 umgesetzt und vorgestellt (und die Macher des „Microsoft Vision“-Werbevideos inspiriert?): NAVI – Navigational Aids [...]
[...] fans to tinker with the hardware and come up with fascinating projects. Notable examples include a navigational device for the visually impaired and a system for providing surgeons with sensory feedback during remote [...]