A string connecting people and devices, about input devices

The iPhone has changed the way we communicate.

On January 9, 2007, Apple CEO Steve Jobs introduced a new device and the world of mobile changed forever. most innovative With the appearance of one of the inventions, the iPhone, the era of the smartphone began.

You can access online anytime and anywhere, play games on the bus to work, take pictures of lunch menus, and buy things at department stores with mobile applications. 

• 

And Steve Jobs said we were born with the ultimate pointing device, and the iPhone uses our fingers to create the most innovative user interfaces. No stylus neededsaid. Since then, the touch screen has become the basic form of smart phones. And he predicted the disappearance of the stylus.

Survival of the unexpected stylus

Jobs recalled: He said that Microsoft would completely change the world with tablet PC software, so notebook computers would disappear completely. So Apple kept pestering me about getting licenses for the Microsoft software they developed.

But his device was using the completely wrong method. It came with a stylus. If you have a stylus, that's it. That evening he must have told me about it ten times. I was fed up and came home and said: “You are laughing. I'll show you what a real tablet is like.”

The next day, Jobs went to work, gathered his team, and talked. “Let's make a tablet computer. However, it must not come with a keyboard or stylus.” It was supposed to be possible to type by touching the screen with your finger. 

In 2002, Jobs was annoyed by Microsoft engineers who were constantly rambling about software for a tablet computer he had developed. It was a way of entering information on a screen using a stylus or pen.

Several manufacturers released tablet PCs using such software that year, but none of them left a mark on the universe. Jobs was maddening to show him the right prototype of a PC (don't use a stylus!).

Source: <Steve Jobs>, quoted by Walter Isaacson, Minumsa, 2011. 738p, 773p

• 

Apple, which had argued for the uselessness of the stylus, also released the Apple Pencil along with the iPad Pro in 2015 under the leadership of Tim Cook after Jobs passed away from pancreatic cancer.

There were controversies such as overturning the theory of the uselessness of the pen, problems with the writing feeling, and separate sales, but there were also things that the pen (stylus) and physical keyboard could do better than the ultimate pointing device, the finger. Users who used to type documents with a keyboard or draw with a pencil or tablet have already familiar input deviceIt was a natural result to find .

• 

Even with such advanced devices, If there is no suitable input device Its value will decrease and there will be limits to what users can do with their devices. The advent of the touch screen was truly revolutionary. But using your fingers to draw is very inefficient. Today, I would like to explain the input device of the XR device.

XR devices and input devices

VR devices and AR devices will have different types and uses of input devices depending on their respective uses, but the types are so diverse that today, based on the chart below, I will classify input devices and explain simple usage examples. As you can see from Type on the left It is based on visual and auditory information, motion, position and direction.

• 

visual based

Normal Camera (Default Camera)

It is the most common input method and is used as an input method for various XR contents. In terms of recognizing objects and patterns, marker-based augmented reality can be a good example. It is often used in combination with other methods.

• 

• 

• 

• 

Infrared Camera

Unlike general cameras, it is literally an input method using an infrared camera. Microsoft's game console XBOX and Windows PC's peripheral device Kinect are the most famous infrared-based input devices.

It can recognize voice along with the user's motion, and can grasp the shape and motion of an object more accurately than a general camera by using infrared light. The video below is an Azure Kinect (2019) product developed for industrial use and shows an example of detecting objects.

ear base

Microphone

Like a normal camera, the device's built-in camera can recognize voices and issue commands. Although sometimes used alone, microphones are often used in conjunction with other device functions.

• 

movement base

As a basis for detecting motion, it is one of the most widely used input devices. Devices commonly referred to as controllers, as well as mice and keyboards, fall into this category. Data can be entered by moving the controller, clicking buttons, or typing.

And you can draw a picture or write a text accordingly. previously mentioned in another article tilt brushcan be a representative example, and the sensor in the controller can measure the movement by calculating location information and acceleration.

• 

• 

And motion-sensing based devices are mainly used for gesture interaction and motion capture. It is also used with computer graphics to produce movies, games, and animations.

location base

Global Positioning System (GPS), Radio Frequency Identification (RFID)

GPS is a technology we are all too familiar with, and it is a satellite navigation system developed and managed by the United States, and it allows us to accurately know our location by communicating with satellites anywhere in the world. Location information consists of longitude, latitude, and elevation (altitude above sea level). In addition, accurate time information can be obtained.

RFID is unfamiliar, but it is a more familiar technology than you might think. A technology that uses radio waves to recognize information at a distance. It is also called RF for short. NFC technology used for smartphone payments is similar to RFID. However, there is a difference in that the reception range of RFID is longer and one-way communication (NFC is two-way communication).

direction base

Gyroscope, Magnetic Sensor

Gyroscope sensors and magnetic sensors are likewise used with the other input devices mentioned above. First, the gyro sensor is used to measure the moving direction in X, Y, and Z coordinates. And by processing this output signal, it is possible to inversely estimate and inversely calculate dynamic forces such as object acceleration, vibration, and impact.

• 

Inside the magnetic sensor, sensors that can measure the strength of the magnetic field are attached in the X, Y, and Z-axis directions like the gyro sensor, and the direction of the magnetic field is measured by summing the outputs of each sensor. In this way, it is used as a digital compass by measuring the earth's magnetic field, and a location-based service is implemented in combination with a GPS sensor.

• 

concluding remarks

As technology develops, more input devices are appearing. And the same applies to the field of XR, and a small birth that sparks innovation can start with a single line of a penciled sentence. Looking forward to more input devices that will appear in the future, you can imagine what the future will look like.