Chapter 12. Machine Learning
In this introduction to the Cloud AI Adventures series, Yufeng Guo demystifies machine learning (ML), moving it away from the realm of "magic" and defining it simply as using data to answer questions.
The summary of his talk is broken down into these core themes:
The Necessity of Machine Learning
As the global volume of data from humans and devices surpasses our ability to manually analyze it, automated systems have become essential.
Scale: Humans can no longer write manual rules fast enough to keep up with data growth.
Adaptability: ML systems excel at learning from shifting landscapes and changing data patterns.
Applications and Impact
Machine learning is already deeply integrated into modern life, often behind the scenes.
Common Uses: Photo tagging, recommendation engines (like YouTube), and fraud detection.
Core Example: Google Search uses ML to understand intent, such as distinguishing whether a search for "Java" refers to coffee or programming.
Specialized Fields: It is revolutionizing healthcare (cancer detection), retail, and autonomous transportation.
The Two Pillars of ML
Guo simplifies the technical process into two distinct phases:
Training (Using Data): The process of using existing data to create and fine-tune a predictive model.
Inference (Answering Questions): Using that trained model to make predictions on new, unseen data.
Key Takeaway: We are moving toward a future where "personalized, insightful, and self-correcting" technology is an expected standard rather than a luxury. To participate, organizations simply need data, developers, and a willingness to start.
Machine learning in a pure form can theoretically go back to Bayes Theorem in the late 1700’s which provided rules for researchers the probability of a cause given analysing the effect. Fast forward to the 1940’s where the first Neural Network model was introduced focused on modeling the brain as mathematical neurons.In the 1950s IBM leveraged these models to play checkers which proved that these models have the ability to learn.
During the 80’s and 90’s introduced capabilities Bayes' theorem gives a mathematical rule for inverting conditional probabilities.
like speech synthesis to pronounce words and Long Short-term Memory allowing models to remember information.
In modern times companies like Google developed functionalities that have made machine learning so useful and successful.
What is Machine Learning?
Machine Learning is a foundational technology that has been a major contributor to modern society. As a Functional Tool it uses learned knowledge to provide solutions for real world problems.
It can enable cars to see and navigate roads and help medical teams diagnose diseases that the naked eye might miss.
With traditional software a developer would code certain instructions to obtain a desired output. In machine learning goals and outputs are provided and the machine provides the logic to obtain the output or goal.
It is accomplished through observations or training of data. In general the more data that the machine learning model is trained on the better or more accurate the results are.
If we look at where machine learning fits in relation to Artificial intelligence, AI is the high level concept of using computers to mimic the intelligence of humans whereas Machine Learning is the technology used to execute the goals.
ML models can be generally looked at as to perform two main functions.:
Classification, where machine learning can identify or categorize data. Think of cancer detection where learning from millions of cancer cells can better diagnose the disease.. Prediction, is when ML models study patterns say of stock movements or weather patterns and can predict or intelligently state what outcome will occur.
How it works.
To be effective Machine learning models need to consume large volumes of data. This data also must be as accurate as possible meaning bad data must be excluded when possible.
Once the data is ingested it needs to be analyzed for data issues. Bad or missing data can affect the model's outcome and accuracy. Identifying outliers or data that is so out of line with the rest of the batch is important in the analysis phase.
Next the process will look for patterns known as the training phase. This phase allows the model to understand the data. And create relationships between inputs and outputs. Also, the process involves iterating through the data making guesses and reviewing those guesses to see if they were correct. If wrong, the model will adjust the process.
The Make Predictions phase is where the trained model provides answers. The prediction normally will come with a probability of the accuracy. In classification, when looking at sentiment using NLP, the model may give you a 80% chance of being positive.. In regression, the model may product a specific value.
The last phase to send answers known as the Labeling or Supervised Learning phase provides the predictions .
identifies patterns in data to create its own logic, its ability to improve performance through experience and statistical modeling.
Data is transformed into "features" and split into training (to build the model) and testing (to validate accuracy) sets.
Modeling: An algorithm is selected—ranging from simple regressions to complex Neural Networks—and trained to minimize errors.
Output: The final result is a model file that can take new, unseen inputs and generate accurate predictions or decisions.
The Three Pillars of ML
Learning typically falls into one of three categories. Supervised Learning is where the model learns from a labeled data set. The input and correct answer to perform tasks like classification or regression.
Unsupervised Learning: The model analyzes "unlabeled" data to find hidden structures, such as clustering similar customers together.
Reinforcement Learning: An agent learns through trial and error, receiving rewards or penalties to optimize its actions (e.g., robotics or gaming).
What it isn't: Static spreadsheet formulas, basic file-moving scripts, or "Expert Systems" that rely on thousands of hard-coded "if-then" rules.
What it is: A system that can identify a trend in one dataset and successfully apply that logic to a completely new piece of information.
Neural Networks
A Neural Network is a model designed by the structure of the human brain. It can recognize patterns, interpret data and solve complex problems by duplicating the way the brain uses neurons to send signals to one another.
A neural network has three distinct types of layers:
The Input Layer where the model will receive data points.
Hidden Layers where processing of the data happens.
Output Layer where the prediction is made either by classification or regression.
How it Works.
It used Nodes to act like processing units. They receive signals from previous layers, process the information and decide whether to pass the information forward. The Neurons act as processing units. Nodes receive signals from the previous layer then process them and decide whether to pass the signal forward.
Every connection between nodes has a weight which is the measurement of the importance of the information. The higher the weight the more influential the data is to the model. The neuron receives inputs from the previous layer and the input is multiplied by the weight.
Biases are values added to nodes to which help with fitting the data. The bias value is added to that sum to shift the result and give the network more flexibility.
Neural networks learn and it loops through a process of trial and error.
Forward Propagation allows data to move from input to output so the network can make a guess.
The Training adjusts weights to minimize error.
CPU, GPU and TPU
CPU or Central Processing Unit is a core concept of computing being it is the chip or chips that handle the processing inside a computer.
GPU stands for Graphics Processing Unit and can handle thousands of small tasks simultaneously. They work in parallel and dramatically reduce the training time of data. Because they were originally designed for gaming and the intense graphics needed to build these systems hence graphical is used in the name but these chips do not specifically handle just graphics processing.
TPU or Tensor Processing Unit are ASICs (Application-Specific Integrated Circuits) created by Google to accelerate the mathematical processing needed for deep learning.
Google Cloud Machine Learning Platform.
Pre-trained Machine Learning APIs like Vision API, Natural Language API,: Translation, Speech-to-Text, and Text-to-Speech.
Automated Machine Learning (AutoML) allows for a simple process to handle complex machine learning problems. It provides automated training for tabular, image, text, and video data via a graphical user interface that is easy-to-use and navigable.
It handles data preprocessing functions like cleaning data and handling missing values. Allows for selecting most relevant variables from data for model accuracy. Access to multiple algorithms and the fine tuning and choosing of those algorithms to maximize performance.
Labeled data is provided and the Google AutoML platform builds a model and provides predictions.
Vertex AI Custom Training process allows developers to leverage commonly used frameworks like TensorFlow, PyTorch, or scikit-learn. Developers can have control over the models architecture, training logic and libraries.
Applications can be packaged into containers and Vertex AI provisions the infrastructure and executes the applications code.
Modern breakthroughs
Machine learning has been a major contributor to advancing society and helping people live longer, live better and help people with disabilities improve their lives.
Some of these issues were too complex for humans to solve without the help of AI and technology.
Here are some of the major advancements in society that machine learning has contributed to.
Developed by researchers at Massachusetts General Hospital and MIT a lung cancer detection process which they named Sybill addresses a gap in oncology.
Sybill can detect lung cancer before it becomes Stage 4 while it can still be treated. Stage 4 is the most advanced stage where the disease has spread from its original site The technology analyzes patterns in CT scans and can predict if a patient will develop the cancer. It has a 94% accuracy and predicts where the tumor will appear before it is visible.
often detecting markers for oncology or cardiovascular issues before physical symptoms appear.
conditions faster and more reliably.
Genentech, along with NVIDIA, is leveraging Generative AI to help in advancing new breakthroughs in pharmaceutical solutions.
They are using a method they are calling “Lab in the Loop."
The strategy focuses on developing an interactive relationship between biology and computer science.
Algorithms based on trained data are used to design new molecules which are tested in a lab. The results are fed back into the AI process which improves the results. Continues this cycle until design goals are met and can be used for clinical medicine.
AI is critical in the development of tools to help individuals with disabilities improve their lives and become more independent.
Advances in the areas of communication, mobility, and sensory perception, have improved the quality of life for people with disabilities>
Sensory and communication advancements allowed for real-time object recognition and automated text delivery for digital images.
Speech-to-text, automated captioning, and sign language translation helps with auditory issues
Natural Language Processing assists people who have trouble speaking.
People who have lost limbs or can’t walk can move better with the advancements in AI developed mobility devices..
Artificial Intelligence is allowing the world to get warnings when natural disasters will and where they will occur.
AI can analyze satellite imagery and atmospheric pressure and predict weather patterns and predict earthquakes giving people time to prepare and evacuate.
Monitoring ocean data can detect flood zones and tsunamis.
Heat signatures, drought conditions, and wind speeds can predict wildfires.
Companies and organizations have developed AI tools for detection. Google has developed a Flood Forecasting System that predicts floods days in advance in high-risk areas. NASA has a Deep Learning Seismic Network that provides faster alerts by identifying earthquake patterns. IBM’s Watson Climate AI helps governments with evacuations.
No comments:
Post a Comment