*1 Saitama Univ. Graduate School of Science and Engineering
Shimo-okubo 255, Sakura-ku, Saitama, 338-8570 Japan
*2 École Polytechnique Fédérale de Lausanne (EPFL) School of Engineering (STI)
Lausanne, CH-1015 Suisse
In recent years, computers are used widely for information gathering with the increasing use of smaller devices, such as mobile phones, smartphones, and tablet computers. Information gathering by computers can be useful not only for non-handicapped people but also for handicapped people in their social participation. At present, the global population of visually-impaired people is approximately 285 million, while their number being at least 370,000 in Japan (1) (2). Visually-impaired people are categorized into totally blind people and weak-sighted people with residual vision, and the proportion of weak-sighted people accounts for more than 60% of all visually-impaired people in Japan (2). Furthermore, there are quite a few former sighted people who became visually impaired due to illness or injury. According to the survey by the Ministry of Health, Labour and Welfare, approximately 10% of the visually-impaired people including weak-sighted people currently use computers, and approximately 10% of the people who have not used computers answered that they wish to use them (3). According to a survey on the use of computers by visually-impaired people, various problems have been pointed out with regard to their use of computers for information gathering (4). Currently, computers generally operate in the GUI (Graphical User Interface) environment. Since visually-impaired people have difficulties obtaining visual information, it is difficult for them to use such computers. Especially, in the case of acquired visual impairment, it is necessary to undergo rehabilitation since the operation environment has markedly changed.
In this study, we examined the possible use of a commercially available inexpensive haptic interface device for guiding the cursor in the GUI environment. Using such a system, we aimed to provide computer operation assistance for visually-impaired people, especially weak-sighted people, to help them use computers in an environment comparable to the GUI environment for sighted people.
In this study, we aimed to establish an interface that can be more readily introduced as an interface for visually-impaired people. As the haptic interface device, we used Falcon produced by a US company Novint Technologies (5). Falcon is a commercially available inexpensive haptic interface device for playing videogames. It is capable of performing force feedback through the operation of the spherical grip connected to its arm. Its grip can be moved three-dimensionally and it can provide feedback of force up to 10 N, sufficient force feedback for the operational assistance for visually-impaired people. It is connected to the computer via USB. Furthermore, various open-source application program interfaces (API) are available for supporting the development of software using Falcon. This makes it possible to easily draw three-dimensional spaces to provide force feedback using 3D Graphics APIs,
Fig. 1 Computer desktop
Fig. 2 Virtual work-space operated by Falcon
such as DirectX and OpenGL, in a C language-based development environment. Thus, we used Falcon to examine a more inexpensive and widely available means of operation assistance.
Figure 1 shows a general multi-window GUI environment. Icons of applications and data are displayed on the desktop, the screen of the lowest layer, and the windows of various applications overlap on top of the desktop. By moving the cursor using a mouse or other pointing device, the user can select icons and data as well as move between windows. As pointing devices, touch panels and pen devices, which allow a more intuitive operation, are also widely used lately. The GUI environment has the characteristics of real-time computing and handling a lot of information simultaneously.
This study was conducted on the supposition of providing operation assistance in a general GUI environment, whereby Falcon, a haptic interface device, was used for moving the cursor. We aimed to guide the cursor movement by applying Falcon’s force feedback to it, thus reducing the computer users’ burden on their vision.
We developed software for indicating the cursor movement and effectively providing force feedback. Using Microsoft Visual Studio 2010, we developed the program using Visual C++ and drew the virtual space using DirectX 9.0c. As Falcon’s API, we used HDAL SDK. As the operating environment, we used a commonly used personal computer operating on a Windows operating system. Figure 2 shows
Fig. 3 Multiple-window GUI
the virtual space we produced for Falcon operation using the developed software. It uses the entire desktop as a three-dimensional virtual space. This window of the virtual space is hidden from view during the operation. Each window laid on top of the desktop is displayed in the virtual space as a cuboid. The spherical cursor in the virtual space is synchronized with Falcon’s gripping position, which is synchronized with the position of the cursor on the desktop. If the cursor in the virtual space overlaps with the wall of a cuboid, force feedback is provided for the user to perceive the position and shape (size) of each window. Furthermore, if the cursor overlaps with the inside of a window (the front wall of the cuboid in the virtual space), the force feedback is provided to supply the information on the window.
Figure 3 shows the conceptual diagram in the case of displaying multiple windows. On the desktop, multiple windows overlap each other over the desktop, which is the screen of the lowest layer. The window being operated (active window) is the one of the highest layer. Other windows are at the middle layer. In the virtual space, the innermost wall corresponds with the desktop, and other windows overlap on top of it as cuboids with virtual depth. If a window is selected, it moves to the highest layer on the desktop, while its corresponding cuboid moves up to the front in the virtual space. This allows operation assistance in a multi-window environment.
We conducted an experiment whereby the subjects used Falcon to move the cursor on a desktop with multiple windows, so as to test whether the force feedback corresponding to the cursor position enabled them to perceive the windows as cuboids without relying on their vision. The subjects were 12 sighted people (comprised of 11 males and 1 female in their 20s). On the supposition of use by people with acquired visual impairment, we had the subjects wear sleeping masks during the test. As a result, we observed that the use of Falcon for providing force feedback corresponding to the cursor position enabled the subjects to perceive the windows without relying on their vision. Furthermore, the subjects were able to perceive the position of the windows to a certain degree. The results suggest the effectiveness of a commercially available inexpensive haptic interface device for providing operation assistance for people with acquired visual impairment for using computers in a general GUI environment.
We examined the possible use of a haptic interface device for providing operation assistance for visually impaired people, especially weak-sighted people, in using computers in the multi-window GUI environment. We developed software designed to guide the cursor through the use of a haptic interface device in a virtual space corresponding to the desktop. Using the software, we performed an experiment and observed that it is possible to perceive windows by force feedback. In the future, we will perform experiments involving visually-impaired people to assess the effectiveness of the system. Furthermore, we will examine the additional use of voice information to improve the system usability.
ResearchAdmissionAcademic ProgramsExchange ProgramsStudent Life
© Copyright Saitama University, All Rights Reserved.
Menu
Home
Japanese
Current Students
by User
Contents