Chapter 17. Cloud Translation
An overview of machine translation
How the Cloud Translation API works
How Cloud Translation pricing is calculated
An example of translating image captions
Starts out easy with vocabulary problems where you memorize the foreign
equivalent of a word you know.
Memorizing a simple map from language A to language B
(for example, houseInSpanish = spanish['house'])
Challenging for humans, computers are good at it.
Memorization problem is nowhere near as challenging as a true understanding of the language.
“conceptual representation” in one language and translate it to another.
Languages started with simple words and grew from there, evolving over hundreds of years into the languages.
Languages, such as Japanese, feature extraordinarily high levels of complexity
17.1. How does the Translation API work?
Google’s Neural Machine Translation
How to resolve the complicated problem of understanding vocabularies and
grammatical rules.
Teach the computer all of the different word pairs
(for example, EnglishToSpanish('home') == 'casa').
Language is extraordinarily complex, with exceptions for almost every rule,
but it is constantly evolving.
Another way (and the way that Google Translate uses for many languages)
uses something called statistical machine translation (SMT).
SMT relies on the same concept as the Rosetta stone, which was a stone
engraved with the same inscription in both ancient Greek and Egyptian hieroglyphics.
The Algorithm relies on millions of documents that have equivalents in at
least one language pair (for example, English and Spanish).
Identifies common patterns across the two documents that are unlikely to be coincidence.
These patterns occur often, it’s likely a match between a phrase in the original text and the equivalent phrase in the translated text.
You see the word “house” over and over in the English translation,
and the word “casa” in the Spanish appears with similar frequency.
As you see more and more of this pattern (with matching occurrences of
“house” in English and “casa” in Spanish).
Continue to train your system on these new inputs and it identifies more and more patterns.
Translating “I went to the store” from English to Spanish comes across fairly well.What about a more complex sentence?
Probleme kann man niemals mit derselben Denkweise lösen,
durch die sie entstanden sind.
German to English comes out as,
“No problem can be solved from the same consciousness that they have arisen.”
Google to focus on some newer areas of research,
including the same technology underlying the Natural Language API and the Vision API:
neural networks.
Google’s Neural Machine Translation (GNMT) system relies on a neural network, uses custom Google-designed and -built hardware to keep things fast.
Neural networks are a set of algorithms, modeled loosely after the human brain, that are designed to recognize patterns.
They interpret sensory data through a kind of machine perception,
labeling or clustering raw input.
That same sentence in German becomes much more readable:
“Problems can never be solved with the same way of thinking that caused them.”
17.2. Language detection
Translation API is looking at some input text and figuring out what language it is.
The Cloud Translation API is completely stateless, meaning that you store nothing and send all the information required in a single request.
Enable the Translation API
See something that looks like the following:
>Sentence: "Probleme kann man niemals mit derselben Denkweise lösen, durch die sie entstanden sind."
- Language: de
- Confidence: 0.832740843296051
Sentence: "Yo soy Americano."
- Language: es
- Confidence: 0.26813173294067383
Important for our purposes, the detections were accurate.
Confidence level is associated with the result.
17.3. Text translation
Given some input text and a target output language, the API will return the translated text .
Translating text is stateless as well.
The language you want along with the input text—you don’t specify the language of the input text.
Output with the sentences translated:
No problem can be solved from the same consciousness that they have arisen.
I am American.
No confidence score associated with the translation.
In another language it might vary depending on who’s doing the translating. Thanks to this ambiguity, a confidence rating wouldn’t be that useful.
Look at the raw API response, which shows the detected language.
Should see output that shows the detected languages:
Sentence 1 was detected to be de
Sentence 2 was detected to be es
17.4. Understanding pricing
Translation API you pay for only what you use.
You’re charged based on the number of characters you send to the API (at a rate of $20 per million).
Given a character is multiple bytes (such as a Japanese character), you’re only charged for that one character.
Whitespace characters are necessary to understand the breaks between words, so they are charged like any other character (or code point).
17.5. Case study: translating InstaSnap captions
Breaking the problem down a bit more, you want to detect if the language of a given caption isn’t the same as the user’s language. I
automatic translation at view-time.
It’s unlikely that every one of the captions needs to be translated.
Assume that you know the primary language of each user because they chose one when they signed up for InstaSnap.
Detect language at “post time,” you can compare it to the viewer’s language.
Summary
Machine translation is the way computers translate from one language to another, with as much accuracy as possible.
Until recently, most translation was done using mappings between languages, but the quality of the translations can sometimes seem a bit “mechanical.”
Cloud Translation is a hosted API that can translate text between languages using neural machine translation, a specialized form of translation that uses neural networks instead of direct mappings between languages.
Cloud Translation charges prices based on the number of characters sent to be translated, where a character is defined as a code point.
No comments:
Post a Comment