Google Lens is one of the company’s most useful and impressive tools: With it, you can scan objects in the world around you to learn more about anything from plants to pets, as well as lift text right out of an image.
For example, pointing your phone’s camera at a poster with text written in a language you don’t know will see Google Lens translate it right before your eyes. It will also read the text out loud if you so wish.
It’s easy to use Google Lens, but how does it work?
How does Google Lens work?
“Part of what my team does is get Lens to recognise not just the text, but also the structure of the text,” Google software engineer Ana Manasovska explains in a blog post.
“We humans automatically understand writing that is separated into sentences and paragraphs, or blocks and columns, and know what goes together.”
The problem is, it’s very difficult for a machine to distinguish that.
Essentially, what Manasovska described is machine learning.
Google Lens uses systems, known as models, that have been trained to discern characters and structure in images.
“A traditional computing system would have only a limited ability to do this. But our machine learning model has been built to ‘teach itself’ on enormous datasets and is learning to distinguish text structures the same way a human would,” she continues.
What can you use Google Lens for?
Say for instance you’re in a café and you like a specific vase they have – you can aim your camera (with Lens installed) at the vase and use Lens to find similar items. Another example is if you don’t know what plant you’re currently looking at, or even breed of dog – almost anything, it Google Lens will You can also use it for translating text, as Manasovska mentioned above, but this extends to help with homework. Pointing Lens at a math problem, for example, will see Google return a bunch of videos/explainers on how to solve that specific problem.
How to use it
Google Lens has been available on non-Google smartphones for a few years. To use it on a non-Google phone, however, you need to download the app. Once in the app, you couldn’t possibly go wrong. You can also use it via Chrome. In the Images tab, when an image from your search results is clicked on, you’ll see the lens icon on the bottom left. You can also right click to select ‘Search image with Google’ and similar images will be displayed.